chore(src): link all scripts to code base and update README.

Author: Kin-Zhang

Dockerfile

@@ -9,6 +9,8 @@ DeFlow: Decoder of Scene Flow Network in Autonomous Driving
 
 Task: Scene Flow Estimation in Autonomous Driving. 
 
+📜 2024/07/24: Merging all scene flow code to a codebase to update one general repo only. This repo will save DeFlow README and [cluster slurm files](assets/slurm).
+
 🤗 2024/11/18 16:17: Update model and demo data download link through HuggingFace, personally I found that `wget` from the HuggingFace link is much faster than Zenodo.
 
 📜 2024/07/24: Merging SeFlow & DeFlow code together, lighter setup and easier running.
@@ -18,92 +20,46 @@ Task: Scene Flow Estimation in Autonomous Driving.
 Pre-trained weights for models are available in [Zenodo](https://zenodo.org/records/13744999)/[HuggingFace](https://huggingface.co/kin-zhang/OpenSceneFlow).
 Check usage in [2. Evaluation](#2-evaluation) or [3. Visualization](#3-visualization). 
 
-**Scripts** quick view in our scripts:
-
-- `dataprocess/extract_*.py` : pre-process data before training to speed up the whole training time. 
-  [Dataset we included now: Argoverse 2 and Waymo, more on the way: Nuscenes, custom data.]
-  
-- `train.py`: Train the model and get model checkpoints. Pls remember to check the config.
-
-- `eval.py` : Evaluate the model on the validation/test set. And also output the zip file to upload to online leaderboard.
-
-- `save.py` : Will save result into h5py file, using [tool/visualization.py] to show results with interactive window.
-
-
-<details> <summary>🎁 <b>One repository, All methods!</b> </summary>
-<!-- <br> -->
-You can try following methods in our code without any effort to make your own benchmark.
-
-- [x] [SeFlow](https://arxiv.org/abs/2407.01702) (Ours 🚀): ECCV 2024
-- [x] [DeFlow](https://arxiv.org/abs/2401.16122) (Ours 🚀): ICRA 2024
-- [x] [FastFlow3d](https://arxiv.org/abs/2103.01306): RA-L 2021
-- [x] [ZeroFlow](https://arxiv.org/abs/2305.10424): ICLR 2024, their pre-trained weight can covert into our format easily through [the script](tools/zerof2ours.py).
-- [ ] [NSFP](https://arxiv.org/abs/2111.01253): NeurIPS 2021, faster 3x than original version because of [our CUDA speed up](assets/cuda/README.md), same (slightly better) performance. Done coding, public after review.
-- [ ] [FastNSF](https://arxiv.org/abs/2304.09121): ICCV 2023. Done coding, public after review.
-<!-- - [ ] [Flow4D](https://arxiv.org/abs/2407.07995): 1st supervise network in the new leaderboard. Done coding, public after review. -->
-- [ ] ... more on the way
-
-</details>
-
-💡: Want to learn how to add your own network in this structure? Check [Contribute](assets/README.md#contribute) section and know more about the code. Fee free to pull request!
-
 ## 0. Setup
 
-**Environment**: Clone the repo and build the environment, check [detail installation](assets/README.md) for more information. [Conda](https://docs.conda.io/projects/miniconda/en/latest/)/[Mamba](https://github.com/mamba-org/mamba) is recommended.
+**Environment**: Clone the repo and build the environment, check [detail installation](./OpenSceneFlow/assets/README.md) for more information. [Conda](https://docs.conda.io/projects/miniconda/en/latest/)/[Mamba](https://github.com/mamba-org/mamba) is recommended.
 
 
 ```bash
-git clone --recursive https://github.com/KTH-RPL/DeFlow.git
-cd DeFlow
+git clone --recursive https://github.com/KTH-RPL/OpenSceneFlow.git
+cd OpenSceneFlow
 mamba env create -f environment.yaml
 ```
 
 CUDA package (need install nvcc compiler), the compile time is around 1-5 minutes:
 ```bash
-mamba activate deflow
+mamba activate opensf
 # CUDA already install in python environment. I also tested others version like 11.3, 11.4, 11.7, 11.8 all works
 cd assets/cuda/mmcv && python ./setup.py install && cd ../../..
 cd assets/cuda/chamfer3D && python ./setup.py install && cd ../../..
 ```
 
-Or another environment setup choice is [Docker](https://en.wikipedia.org/wiki/Docker_(software)) which isolated environment, you can pull it by. 
-If you have different arch, please build it by yourself `cd DeFlow && docker build -t zhangkin/seflow` by going through [build-docker-image](assets/README.md/#build-docker-image) section.
-```bash
-# option 1: pull from docker hub
-docker pull zhangkin/seflow
+Or another environment setup choice is [Docker](https://en.wikipedia.org/wiki/Docker_(software)) which isolated environment, check more information in [OpenSceneFlow/assets/README.md](./OpenSceneFlow/assets/README.md#docker-environment).
+
 
-# run container
-docker run -it --gpus all -v /dev/shm:/dev/shm -v /home/kin/data:/home/kin/data --name deflow zhangkin/seflow /bin/zsh
-# then `mamba activate seflow` python environment is ready to use
-```
 
 ## 1. Run & Train
 
 Note: Prepare raw data and process train data only needed run once for the task. No need repeat the data process steps till you delete all data. We use [wandb](https://wandb.ai/) to log the training process, and you may want to change all `entity="kth-rpl"` to your own entity.
 
 ### Data Preparation
 
-Check [dataprocess/README.md](dataprocess/README.md#argoverse-20) for downloading tips for the raw Argoverse 2 dataset. Or maybe you want to have the **mini processed dataset** to try the code quickly, We directly provide one scene inside `train` and `val`. It already converted to `.h5` format and processed with the label data. 
+Check [OpenSceneFlow/dataprocess/README.md](./OpenSceneFlow/dataprocess/README.md#argoverse-20) for downloading tips for the raw Argoverse 2 dataset. Or maybe you want to have the **mini processed dataset** to try the code quickly, We directly provide one scene inside `train` and `val`. It already converted to `.h5` format and processed with the label data. 
 You can download it from [Zenodo](https://zenodo.org/records/13744999/files/demo_data.zip)/[HuggingFace](https://huggingface.co/kin-zhang/OpenSceneFlow/blob/main/demo_data.zip) and extract it to the data folder. And then you can skip following steps and directly run the [training script](#train-the-model).
 
 ```bash
 wget https://huggingface.co/kin-zhang/OpenSceneFlow/resolve/main/demo_data.zip
 unzip demo_data.zip -p /home/kin/data/av2
 ```
 
-#### Prepare raw data 
-
-Checking more information (step for downloading raw data, storage size, #frame etc) in [dataprocess/README.md](dataprocess/README.md). Extract all data to unified `.h5` format. 
-[Runtime: Normally need 45 mins finished run following commands totally in setup mentioned in our paper]
-```bash
-python dataprocess/extract_av2.py --av2_type sensor --data_mode train --argo_dir /home/kin/data/av2 --output_dir /home/kin/data/av2/preprocess_v2
-python dataprocess/extract_av2.py --av2_type sensor --data_mode val --mask_dir /home/kin/data/av2/3d_scene_flow
-python dataprocess/extract_av2.py --av2_type sensor --data_mode test --mask_dir /home/kin/data/av2/3d_scene_flow
-```
-
 ### Train the model
 
-All local benchmarking methods and ablation studies can be done through command with different config, check [`assets/slurm`](assets/slurm) for all the commands we used in DeFlow raw paper. You can check all parameters in [conf/config.yaml](conf/config.yaml) and [conf/model/deflow.yaml](conf/model/deflow.yaml), **if you will set wandb_mode=online**, maybe change all `entity="kth-rpl"` to your own account name.
+All local benchmarking methods and ablation studies can be done through command with different config, check [`assets/slurm`](assets/slurm) for all the commands we used in DeFlow raw paper. You can check all parameters in [OpenSceneFlow/conf/config.yaml](./OpenSceneFlow/conf/config.yaml) and [OpenSceneFlow/conf/model/deflow.yaml](./OpenSceneFlow/conf/model/deflow.yaml), **if you will set wandb_mode=online**, maybe change all `entity="kth-rpl"` to your own account name.
 
 Train DeFlow with the leaderboard submit config. [Runtime: Around 6-8 hours in 4x A100 GPUs.] Please change `batch_size`&`lr` accoordingly if you don't have enough GPU memory. (e.g. `batch_size=6` for 24GB GPU)
 ```bash