Need to design ROIs recursively on a project with >1000 video

[DavidBarriere]

Dear SIMBA creator.
Thank you for your work on SIMBA.
I have a big project with more than 1000 videos acquired in standardized conditions (open field).
I leave the GUI and I work on jupyter notebook to process my data.
As far everything works well, however I am puzzled for ROI creation.
Indeed, in some videos, the camera was tilted and some objects were not placed consistently at the same place.

So I'd like to draw my ROIs on to one video and then open recursively each video to move/replace/resize ROIs to make an accurate quantification.
As a strating point I tried on a Jupyter-Notebook to use ROI_define and path both config_path and video_path but :
"""
In [26]: config_path = '/home2/Projet_NAOMI/Behavorial_Analysis/01-SIMBA/NAOMI/p
...: roject_folder/project_config.ini'

In [27]: video_path = "/home2/Projet_NAOMI/Behavorial_Analysis/00-RawData/02-Vid
...: eos/"

In [28]: ROI_define(config_path,video_path)

---

TypeError Traceback (most recent call last)
in
----> 1 ROI_define(config_path,video_path)

TypeError: 'module' object is not callable

And I tried as I found on website :

In [29]: _ = ROI_define(config_path,video_path)

TypeError Traceback (most recent call last)
in
----> 1 _ = ROI_define(config_path,video_path)

TypeError: 'module' object is not callable
"""

If it's not a huge job for you could you give me an example of function or at least a path to write my function to :
1/ open the ROI Gui to draw ROIs on the first video found,
2/ save ROIs in a specific place
3/ Load second video and previous ROIs for correction,
4/ save ROIs in a specific place
etc...

Thank you so much

[sronilsson]

Hi @DavidBarriere - thanks for the feedback, appreciate it!

Just to re-iterate and confirm so I get this right before I give you a notebook solution:

You have >1k videos, and you want to (i) draw the ROIs on one video, (ii) duplicate these ROIs on the subsequent videos, and (iii) iteratively go through the ROIs on the subsequent videos and correct the ROIs.

While there is a method to do this in the GUI (by clicking the the APPLY TO ALL button in the ROI interface:

.... I can se this becoming tricky in the GUI (having to navigate and scroll though 1000 videos). I will get a notebook together to run this. However, just a heads up, manually opening and correcting 1000s videos can take some time and things could go astray, and it may be best to do e.g., 5 batches of 200 videos..

[DavidBarriere]

Dear Simon thank you for your rapid reply.
Yes I saw the button to replicate ROIs and yes Simba crash during the process. However some object are far away to the initial ROI location and when I tried to change ROI location each time it crash
This is the reason of my asking.
You have >1k videos, and you want to (i) draw the ROIs on one video, (ii) duplicate these ROIs on the subsequent videos, and (iii) iteratively go through the ROIs on the subsequent videos and correct the ROIs.'''

Yes It's what I would like to get.

I know that it will be a bit painfull but this is the nature of academic research :)

[sronilsson]

Cheers @DavidBarriere - there seems to be two issues in the GUI I'd like to look into first before dealing with the notebook if that's OK.

You mention it crashes when clicking apply to all button. When this happens, do you see any error messages in the operating system terminal you used to launch Simba from?

Second, I guess the videos have large differences in resolutions, and the ROIs drawn on the first video fall largely outside the frame of other videos, making them hard to select and move? We could insert a button named something like "move to center", which moves any selected ROI in the drop-down to the center of the image when clicked, do you think that could solve it?

[DavidBarriere]

No problem.

When it crash, there is no message. I think it's a memory problem because videos are big.
I work on large animal model (sheep) and video size is large (1920*1080) with a fps = 23s and long (10min).

Videos have been captured under standard conditions in animal facilities with GOPRO camera fixed on the ceil approximately at the same position from the floor. We have 16 facilities and camera have been installed/uninstall after each sessions.
So videos are identical regarding resolution, fps, etc. but objects on the videos (manger/drinker/plateform) are not rigourously at the same position due to multiple installing/uninstalling of the cameras.

I think it's just to huge for the GUI so it's why I try to analyze with SIMBA using command lines.
Importation, interpolation and smoothing passed easily with the given examples.
The rest sounds more tricky :)

[sronilsson]

@DavidBarriere - when you goy a chance, could you give THIS notebook ago? It is also rendered HERE. Please let me know if I got something wrong, or if it could be improved, or if anything is missing.

Check the comments for instructions. And please make sure you are running latest simba with pip install simba-uw-tf-dev --upgrade before testing it.

I tried to write a loop in the final cell, to automatically loop over all the videos. However, the ROI tkinter drawing interface was playing up and locking up when doing this in a jupyter notebook, so at least for this first iteration you'd manually have to update the video_path argument for each video.

Specifically, when completed the _ = ROI_definitions(config_path=PROJECT_CONFIG_PATH, video_path=video_file_paths[1]) set same cell
to _ = ROI_definitions(config_path=PROJECT_CONFIG_PATH, video_path=video_file_paths[2]) and run the same cell again for the next video etc...

[DavidBarriere]

Dear Simon thank you so much for your work.

I tried the code and I have a first error poping :

---

ModuleNotFoundError Traceback (most recent call last)
Input In [1], in <cell line: 3>()
1 from simba.utils.read_write import find_files_of_filetypes_in_directory
2 from simba.utils.enums import Options
----> 3 from simba.roi_tools.ROI_define import ROI_definitions
4 from simba.roi_tools.ROI_multiply import multiply_ROIs

File ~/.local/lib/python3.8/site-packages/simba/roi_tools/ROI_define.py:13, in
10 import PIL.Image
11 from PIL import ImageTk
---> 13 from simba.mixins.config_reader import ConfigReader
14 from simba.mixins.plotting_mixin import PlottingMixin
15 from simba.mixins.pop_up_mixin import PopUpMixin

File ~/.local/lib/python3.8/site-packages/simba/mixins/config_reader.py:36, in
29 from simba.utils.enums import ConfigKey, Defaults, Dtypes, Formats, Keys, Paths
30 from simba.utils.errors import (BodypartColumnNotFoundError, DataHeaderError,
31 DuplicationError, InvalidInputError,
32 MissingProjectConfigEntryError,
33 NoFilesFoundError, NoROIDataError,
34 NotDirectoryError, ParametersFileError,
35 PermissionError)
---> 36 from simba.utils.lookups import (create_color_palettes, get_color_dict,
37 get_emojis, get_log_config)
38 from simba.utils.printing import SimbaTimer, stdout_success
39 from simba.utils.read_write import (find_core_cnt, get_all_clf_names,
40 get_fn_ext, read_config_file, read_df,
41 read_project_path_and_file_type, write_df)

File ~/.local/lib/python3.8/site-packages/simba/utils/lookups.py:15, in
13 import matplotlib.font_manager
14 import pandas as pd
---> 15 import pyglet
16 from matplotlib import cm
18 import simba

ModuleNotFoundError: No module named 'pyglet'
``

[sronilsson]

@DavidBarriere - cheers for testing.

Something seems to have gone wrong with the SimBA environment. I am seeing that you are using python3.8, which SimBA probably could have troubles with. Is this the same SimBA environment you have been using before?

EDIT PS. Try using python 3.6, or if absolutely necessary for whatever reason, python 3.10.

[DavidBarriere]

Okay, I changed the kernel to 3.6 on jupyter and the command run.
I try the rest this morning.
Thank you.

[DavidBarriere]

I think that everything is working but there is some troubles within the GUI display as there is no button displayed and notebook finally crash. Here the terminal :
[W 11:00:35.794 NotebookApp] Notebook SIMBA_Processing.ipynb is not trusted
[W 11:00:35.866 NotebookApp] Trusting notebook /SIMBA_Processing.ipynb
[I 11:00:36.023 NotebookApp] Starting buffering for e1e2c6bb-48bc-404c-bc27-0186b7475fbe:f695e463e47c4fb489436d2d785fde82
[W 11:00:36.331 NotebookApp] 404 GET /nbextensions/pydeck/extension.js?v=20250120105259 (127.0.0.1) 2.100000ms referer=http://localhost:8888/notebooks/SIMBA_Processing.ipynb
[W 11:00:37.336 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20250120105259 (127.0.0.1) 1.200000ms referer=http://localhost:8888/notebooks/SIMBA_Processing.ipynb
[IPKernelApp] ERROR | No such comm target registered: jupyter.widget.control
[IPKernelApp] WARNING | No such comm: 053c3dac-ba55-449f-a6b8-227b9d8ec356
[I 11:01:39.595 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel e1e2c6bb-48bc-404c-bc27-0186b7475fbe restarted

[sronilsson]

Opfhh getting closer! Let me move to a Linux computer and try this a little later, to troubleshoot/ and see if I can replicate - I have only tested on a MS Windows.

[sronilsson]

It did get it to run on a Ubuntu 22.04.4 machine.. :/

JUst a few more questions to see what is going wrong:

If you print out the python version, and the simba version, in the notebook, what do you see as below?

Are your videos/simba project stored on the local machine, or are they located on a server? If a server,would it run on a simple small project that is on the machine, and it freezes up when reaching to the server?

[DavidBarriere]

Hi Simon,
My videos are stored locally on an internal mounted 1To hard disk.
Except the version of python kernel use everything is similar.
I moved to python 3.6.13 and I obtained similar result with ROI gui.

[sronilsson]

Can you test one thing to see if it makes a difference:

The ROIs are stored in a file at project_folder\logs\measures\ROI_definitions.h5. If you delete that file, then restart the notebook and try again, does the interface boot up as expected or not?

[DavidBarriere]

No is still the same unfortunately...

[sronilsson]

Got it - one question: are you using simba through conda or a virtual environment, or is it through base python? If through base python, does it work in conda?

I've tried the notebook on a few machines now, but so far I am failing to recreate the error and trying to figure out what I am doing wrong.

[DavidBarriere]

I am working within a conda env because I need a python3.12 for another application.

Maybe something wrong with Tkinter one my side

[DavidBarriere]

It's almost running outside of jupyter.
When I try in ipython I found button are back. Nevertheless I cannot resize the image of my experiment and neither drawing something because it crash...

[DavidBarriere]

Eventually it's okay !
I think there is something wrong between tk and my jupyter notebook. I will try to fix it.
Thank you for your strong help 🥇. Now I gonna draw :)

[sronilsson]

Ah I think I got it, I was able to replicate the freezing you saw earlier when I initated a jupyter notebook istance outside the IDE on Ubuntu, and I have a potential fix.

[sronilsson]

Oh wow 🚀 didn't see your last message.

If you do encounter issues, please let me know, and I will come back to this!