frigate iX app on TrueNAS Scale with Coral NVMe/PCIe. Can I use it? #9457

MrCaspan · 2024-01-26T22:07:20Z

MrCaspan
Jan 26, 2024

So I was lucky enought to get a PCIe Coral card for my computer. I can see that my TrueNAS scale install is able to see it by doing an 'lspci' and it sows it in the list.

07:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU

When looking at the config for the frigate iX application it seems to only allow the USB version of Coral to be passed through. Is there any way I can get the PCIe verison of it to work IE manually editing the containers config. I know it might require me redo this on every update but I am willing to do this. If anyone could help walk me through what is needed?

Note: I got the PCIe version as I had read many times that the PCIe version is much faster then the USB version. I would really like to use this if possible!

ziferius · 2024-07-19T23:50:36Z

ziferius
Jul 19, 2024

Hey MrCaspan, do you have an update?

7 replies

ElectronicBattle Jul 26, 2024

This might be a red herring because I don't understand well enough but I hope it helps rather than confuses ...

I'm using TN Scale Dragonfish and the iX-application official Frigate App (not the TrueScale one because TS is deprecated now) with the USB Coral and the app specifically says that it doesn't support the m.2 version:

Entirely separate from my USB Coral TrueNAS Frigate app setup, I have a Coral m.2 version plugged into a PCIe adaptor card in my desktop linux box.

My $ lspci similarly says 24:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU which I shall fiddle with at some point (too much fun with the TN USB Coral setup at the moment!).

I get an inference speed with the USB Coral of 9 ms and I am feeding four HD cameras (1080p, 2Mpixels). The various CPU utilisations are low (10-11%). Although the PCIe version might be faster, it might also be an overkill.

scottcondie24 Jul 26, 2024

My Frigate app shows Catalog: Truecharts Train: stable but I have since manually loaded the 0.14 beta4 while testing dropouts. I don't remember seeing that page, maybe it looked different in the truecharts catalog or I forgot in the 14 month gap between setting up and my coral arriving. Sounds like I should test without the coral for a while to see if that's what's causing my issues.

My 'lspci' shows similar
02:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU (prog-if ff)
Subsystem: Global Unichip Corp. Coral Edge TPU

when run 'lsmod | grep apex' it shows
apex 28672 0
gasket 126976 1 apex

9ms is still over 100fps which is plenty for four cameras, my pcie is 6ms for four cameras (so far) so there is plenty of headroom.

ElectronicBattle Jul 26, 2024

on my desktop system $ lsmod | grep apex tells me

apex 28672 0
gasket 122880 1 apex

I don't know what that means but I hope it is helpful for you.

Frigate seems to set the "analysis" video frame rate to 5 Hz == 200 ms which would mean 120 cameras could be processed sequentially at 9 ms assuming linear scaling of frames per second and rtsp stream data and so on.

I've got 8 cameras in total, four enabled and four to fiddle with later when I have got my first F models working (joined today, not requested any yet). So much stuff to work out!

scottcondie24 Jul 26, 2024

Having those in lsmod shows the relevant drivers are installed.

From what I understand if the coral can process 1 frame every 9ms and your detect is set to 5fps then you could have 22 cameras (22x5x9 = 0.99s processing time per second), or you could bump up your fps for 8 cameras but it's diminishing gains.

ElectronicBattle Jul 27, 2024

the relevant drivers are installed.

That's useful to know. I'd failed to get the drivers properly built-in for my OS (python 3 unmet dependencies which were turning into a mess) but it does leave me my preferred option which is Docker / Docker compose (something else which I have vague awareness of but I need to learn!).

could have 22 cameras

That's what I also calculated only I decided to type a "1" in front of the 20 for some reason! Thanks for correcting me.

EB

maxx-ukoo · 2024-07-27T08:45:18Z

maxx-ukoo
Jul 27, 2024

It works for me on truenas scale with coral in M2 slot. I use docker inside jailmaker VM - https://github.com/Jip-Hop/jailmaker
Jail config with all bindings (I have some docker containers with serial ports but for frigate you need apex and dri only)

docker_compatible=1
gpu_passthrough=0
systemd_nspawn_user_args=--bind='/dev/serial/by-path/pci-0000\:00\:14.0-usb-0\:7\:1.0-port0:/dev/ttyUSB0' --bind='/dev/serial/by-path/pci-0000\:00\:14.0-usb-0\:2\:1.0-port0:/dev/ttyUSB1' --bind='/dev/apex_0:/dev/apex_0' --bind='/dev/serial/by-path/pci-0000\:00\:14.0-usb-0\:8\:1.0-port0:/dev/ttyUSB2' --bind='/dev/serial/by-path/pci-0000\:00\:14.0-usb-0\:1\:1.0-port0:/dev/ttyUSB3'

# You generally will not need to change the options below
systemd_run_default_args=--property=KillMode=mixed --property=Type=notify --property=RestartForceExitStatus=133 --property=SuccessExitStatus=133 --property=Delegate=yes --property=TasksMax=infinity --collect --setenv=SYSTEMD_NSPAWN_LOCK=0
systemd_nspawn_default_args=--keep-unit --quiet --boot --bind='/mnt/tb-pool/docker-opt:/opt' --bind='/dev/dri:/dev/dri'`

docker compose:

version: "3.9"
services:
  frigate:
    privileged: true
    container_name: frigate

    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "256mb" # update for your cameras based on calculation above
    devices:
      - /dev/apex_0:/dev/apex_0
      - /dev/dri:/dev/dri
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config.yml:/config/config.yml:ro
      - /opt/frigate/db:/opt/db
      - fr_cifs_mount:/media/frigate

    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
    environment:

      FRIGATE_RTSP_PASSWORD: "********"

volumes:
    fr_cifs_mount:
      driver: local
      driver_opts:
        type: cifs    
        device: "//10.x.x.x/zm"
        o: "username=user,password=pass,iocharset=utf8,file_mode=0777,dir_mode=0777,soft,noperm"

I mount samba share to frigate instead mount volume from nas

frigate config:

mqtt:
  host: mqtt.sng.maxx

database:
  path: /opt/db/frigate.db

ffmpeg:
  hwaccel_args: preset-vaapi
  output_args:
    record: preset-record-generic-audio-aac

record:
  enabled: True
  retain:
    days: 15
    mode: all
objects:
  track:
    - person
    - car
    - bicycle
    - motorcycle
    - cat
    - dog
    - bird
detectors:
  coral:
    type: edgetpu
    device: ""

cameras:
...

3 replies

ElectronicBattle Jul 27, 2024

That's going to be a handy reference for me when I move to Electric Eel/when I get my desktop installation sorted out.

Thanks for posting it!

maxx-ukoo Jul 27, 2024

I have updated comment with correct format.
How achieve it:

install truenas scale via iso, USB, etc
install VM for docker via jailmaker (https://github.com/Jip-Hop/jailmaker). I show my jail config
install portainer (https://www.portainer.io/install) on docker inside VM
install frigate via portainer UI (my frigate config is copy-paste from portainer stack) I show my frigate config with coral TPU
install another services on truenas scale as docker-compose config via portainer (It is nt related to frigate but it can be openhab, influxdb, grafana, gitea and another service. I have it on motherboard from ali on N6005 CPU)

ElectronicBattle Jul 29, 2024

very handy - I'm hoping to try it later this week. Thanks for posting this too.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

frigate iX app on TrueNAS Scale with Coral NVMe/PCIe. Can I use it? #9457

{{title}}

Replies: 2 comments 10 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

frigate iX app on TrueNAS Scale with Coral NVMe/PCIe. Can I use it? #9457

Replies: 2 comments · 10 replies

Replies: 2 comments 10 replies