Art installation with TRAjectory Prediction
Find a file
2025-05-27 17:37:51 +02:00
EXPERIMENTS predict position instead of velocity 2025-01-02 16:24:00 +01:00
trap Video recorder in frame emitter (for gige video) 2025-05-27 17:37:51 +02:00
.gitignore Tools for blacklisting tracks 2024-11-28 16:08:55 +01:00
.python-version Trajectory prediction - test in browser 2023-10-12 20:28:17 +02:00
build_opencv_with_gstreamer.sh Build python-opencv with gstreamer support 2024-11-14 11:37:37 +01:00
custom_bytetrack.yaml Tracker more sensitive and cv_renderer animates lines 2024-12-23 15:35:51 +01:00
pyproject.toml Decoupled monitoring and draw iff lines 2025-05-26 16:14:47 +02:00
README.md Expain trap_rust 2025-05-19 12:05:37 +02:00
supervisord.conf Decoupled monitoring and draw iff lines 2025-05-26 16:14:47 +02:00
test_custom_rnn.ipynb Test with alternative NN for prediction 2025-04-03 20:59:40 +02:00
test_custom_rnn_lstm-attention.ipynb Test with alternative NN for prediction 2025-04-03 20:59:40 +02:00
test_homography.ipynb Map in inference 2024-12-27 16:12:50 +01:00
test_model.ipynb Notebook to visualise predictions of a trained model 2024-12-06 08:29:42 +01:00
test_path_transforms.ipynb Fade effect, and prep for more advanced line corretion using Shapely 2025-04-29 16:58:21 +02:00
test_tracker.ipynb Run tracker with smoother enabled 2024-04-29 14:46:44 +02:00
test_tracking_data.ipynb Notebook updates 2024-12-10 15:43:30 +01:00
test_training.ipynb Fix remote logging 2025-05-19 14:11:56 +02:00
test_training_data.ipynb Map in inference 2024-12-27 16:12:50 +01:00
test_trajectron_maps.ipynb Fixes to config.json for heading derivates and map rendering 2024-12-28 21:02:07 +01:00
uv.lock Fix remote logging 2025-05-19 14:11:56 +02:00

Trajectory Prediction Video installation

Install

  • Run bash build_opencv_with_gstreamer.sh to build opencv with gstreamer support
  • Use uv to install

How to

See also the sibling repo traptools for camera calibration and homography tools that are needed for this repo. Also, trap_rust is used to map the shapes (which are generated by stage.py) to lasers, as to use specific optimization techniques for the paths before sending them to the DAC.

These are roughly the steps to go from datagathering to training

  1. Make sure to have some recordings with a fixed camera. [UPDATE: not needed anymore, except for calibration & homography footage]
    • Recording can be done with ffmpeg -rtsp_transport udp -i rtsp://USER:PASS@IP:554/Streaming/Channels/1.mp4 hof2-cam-$(date "+%Y%m%d-%H%M").mp4
  2. Follow the steps in the auxilary traptools repository to obtain (1) camera matrix, lens distortion, image dimensions, and (2+3) homography
  3. Run the tracker, e.g. uv run tracker --detector ultralytics --homography ../DATASETS/NAME/homography.json --video-src ../DATASETS/NAME/*.mp4 --calibration ../DATASETS/NAME/calibration.json --save-for-training EXPERIMENTS/raw/NAME/
    • Note: You can run this right of the camera stream: uv run tracker --eval_device cuda:0 --detector ultralytics --video-src rtsp://USER:PW@ADDRESS/STREAM --homography ../DATASETS/NAME/homography.json --calibration ../DATASETS/NAME/calibration.json --save-for-training EXPERIMENTS/raw/NAME/, each recording adding a new file to the raw folder.
  4. Parse tracker data to Trajectron format: uv run process_data --src-dir EXPERIMENTS/raw/NAME --dst-dir EXPERIMENTS/trajectron-data/ --name NAME Optionally, smooth tracks: --smooth-tracks
    • Optionally, add a map: ideally a RGB png: 3 layers of 0-255
      • uv run process_data --src-dir EXPERIMENTS/raw/NAME --dst-dir EXPERIMENTS/trajectron-data/ --name NAME --smooth-tracks --camera-fps 12 --homography ../DATASETS/NAME/homography.json --calibration ../DATASETS/NAME/calibration.json --filter-displacement 2 --map-img-path ../DATASETS/NAME/map.png
  5. Train Trajectron model uv run trajectron_train --eval_every 10 --vis_every 1 --train_data_dict NAME_train.pkl --eval_data_dict NAME_val.pkl --offline_scene_graph no --preprocess_workers 8 --log_dir EXPERIMENTS/models --log_tag _NAME --train_epochs 100 --conf EXPERIMENTS/config.json --batch_size 256 --data_dir EXPERIMENTS/trajectron-data
  6. The run!
    • uv run supervisord