Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -104,37 +104,33 @@ git clone https://github.com/YvanYin/Metric3D.git
|
|
| 104 |
```
|
| 105 |
|
| 106 |
## π§± Download Models
|
| 107 |
-
|
| 108 |
To run **MotionAgent**, please download all pretrained and auxiliary models listed below, and organize them under the `ckpts/` directory as shown in the example structure.
|
| 109 |
|
| 110 |
### 1οΈβ£ **Optical Flow ControlNet Weights**
|
| 111 |
-
Download from
|
| 112 |
-
π [Hugging Face (MotionAgent)](https://huggingface.co/leoisufa/MotionAgent)
|
| 113 |
-
and place the files in ckpts/controlnet.
|
| 114 |
|
| 115 |
### 2οΈβ£ **Stable Video Diffusion**
|
| 116 |
-
Download from
|
| 117 |
-
π [Hugging Face (MOFA-Video-Hybrid)](https://huggingface.co/MyNiuuu/MOFA-Video-Hybrid/tree/main/ckpts/mofa/stable-video-diffusion-img2vid-xt-1-1)
|
| 118 |
-
and save the model to ckpts/stable-video-diffusion-img2vid-xt-1-1
|
| 119 |
|
| 120 |
### 3οΈβ£ **Grounding DINO**
|
| 121 |
Download the grounding model checkpoint using the command below:
|
| 122 |
```bash
|
| 123 |
wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
|
| 124 |
```
|
| 125 |
-
Then place it directly under ckpts
|
| 126 |
-
|
| 127 |
-
### 4οΈβ£ **Metric Depth Estimator**
|
| 128 |
-
Download from
|
| 129 |
-
π [Hugging Face (Metric3d)](https://huggingface.co/onnx-community/metric3d-vit-small)
|
| 130 |
-
and place the files in ckpts/metric_depth_vit_small_800k.pth.
|
| 131 |
|
| 132 |
-
###
|
| 133 |
Download the segmentation model using:
|
| 134 |
```bash
|
| 135 |
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
|
| 136 |
```
|
| 137 |
-
Then place it under ckpts
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 138 |
|
| 139 |
After all downloads and installations, your ckpts folder should look like this:
|
| 140 |
|
|
@@ -148,7 +144,9 @@ ckpts/
|
|
| 148 |
```
|
| 149 |
|
| 150 |
## π Running the Demos
|
| 151 |
-
|
|
|
|
|
|
|
| 152 |
|
| 153 |
## π BibTeX
|
| 154 |
If you find [MotionAgent](https://arxiv.org/abs/2502.03207) useful for your research and applications, please cite using this BibTeX:
|
|
|
|
| 104 |
```
|
| 105 |
|
| 106 |
## π§± Download Models
|
|
|
|
| 107 |
To run **MotionAgent**, please download all pretrained and auxiliary models listed below, and organize them under the `ckpts/` directory as shown in the example structure.
|
| 108 |
|
| 109 |
### 1οΈβ£ **Optical Flow ControlNet Weights**
|
| 110 |
+
Download from π [Hugging Face (MotionAgent)](https://huggingface.co/leoisufa/MotionAgent) and place the files in `ckpts`.
|
|
|
|
|
|
|
| 111 |
|
| 112 |
### 2οΈβ£ **Stable Video Diffusion**
|
| 113 |
+
Download from π [Hugging Face (MOFA-Video-Hybrid/stable-video-diffusion-img2vid-xt-1-1)](https://huggingface.co/MyNiuuu/MOFA-Video-Hybrid/tree/main/ckpts/mofa/stable-video-diffusion-img2vid-xt-1-1) and save the model to `ckpts`.
|
|
|
|
|
|
|
| 114 |
|
| 115 |
### 3οΈβ£ **Grounding DINO**
|
| 116 |
Download the grounding model checkpoint using the command below:
|
| 117 |
```bash
|
| 118 |
wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
|
| 119 |
```
|
| 120 |
+
Then place it directly under `ckpts`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 121 |
|
| 122 |
+
### 4οΈβ£ **Segment Anything**
|
| 123 |
Download the segmentation model using:
|
| 124 |
```bash
|
| 125 |
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
|
| 126 |
```
|
| 127 |
+
Then place it under `ckpts`.
|
| 128 |
+
|
| 129 |
+
### 5οΈβ£ **Metric Depth Estimator**
|
| 130 |
+
Download from π [Hugging Face (Metric3d)](https://drive.google.com/file/d/1YfmvXwpWmhLg3jSxnhT7LvY0yawlXcr_/view?usp=drive_link) and place the files in `ckpts`.
|
| 131 |
+
|
| 132 |
+
### 6οΈβ£ **CMP**
|
| 133 |
+
Download from π [Hugging Face (MOFA-Video-Hybrid/cmp)](https://huggingface.co/MyNiuuu/MOFA-Video-Hybrid/resolve/main/models/cmp/experiments/semiauto_annot/resnet50_vip%2Bmpii_liteflow/checkpoints/ckpt_iter_42000.pth.tar) and save the model to `models/cmp/experiments/semiauto_annot/resnet50_vip+mpii_liteflow/checkpoints`.
|
| 134 |
|
| 135 |
After all downloads and installations, your ckpts folder should look like this:
|
| 136 |
|
|
|
|
| 144 |
```
|
| 145 |
|
| 146 |
## π Running the Demos
|
| 147 |
+
```bash
|
| 148 |
+
python run_agent.py
|
| 149 |
+
```
|
| 150 |
|
| 151 |
## π BibTeX
|
| 152 |
If you find [MotionAgent](https://arxiv.org/abs/2502.03207) useful for your research and applications, please cite using this BibTeX:
|