DepthCrafter : new depth map model!

[donrikk]

@nagadomi hello my friend! i just wantedto inform you that the depthcrafter model is out but seems t obe quite annoying to get running locally by hand i was wondering if you could or would add it into iw3 if possible, the license is completely open and ready to use so that shouldnt be a problem at all i just dont know if its possible to implement into iw3 but here is the github page for it and the hugging face page i think has the models and so on.

https://github.com/Tencent/DepthCrafter?tab=readme-ov-file

[loawizard]

Would be great!

[diverswan9]

A sample test from someone somewhere...
https://github.com/user-attachments/assets/1d6bc779-b887-4588-9216-941d77f2b1f3

[nagadomi]

I do not plan to support it in iw3.
If you want to use it, use it yourself.

[mazefish]

Forgive the ignorant question, but how would I go about doing that? I managed to make a depth map with it in ComfyUI but have no idea how to make it into a SBS video. Can it be implemented within the IW3 manually? Any help would be appreciated.

[nagadomi]

See #115

[mazefish]

So far, from trying it out, it does seem better than depth any large. However, not by all that much, and it takes crazy compute by comparison. Is that pretty much what you concluded too?

[nagadomi]

I do not want to write a critique of a research field, so I do not want “model support request” kind of discussion.
Sorry for it.

[nagadomi]

To avoid depending on my personal decision, I think it would be better to develop a SBS stereo node for ComfyUI.

[gituser123456789000]

I haven't been active in 2D to 3D conversions the past few months, so DepthCrafter just recently came to my attention. It does look very impressive as far as temporal consistency and the maps look good (although they need to be seen in actual conversions to make a true judgment). In some examples they appear to be significantly better than Depth-Anything V2. I can't say that every detail looks better, like depth through car windows.. I'm unsure if that's better to push to the background or to be on the same plane as the car door, but several other examples look like clear improvements, either with significantly better background detail, and certainly with consistency from frame to frame.... but it also seems like it's not comparing apples to apples. DepthCrafter seems to be a full program specifically focusing on temporal stability, while Depth-Anything is just creating depth maps for each frame, so it's not really a direct comparison.

First impression seems to be impressive, but the real proof is actually viewing results in 3D.. and if DepthCrafter is limited to realtively low resolution, 5 second only clips with very high PC requirements, that's a big issue as far as usability.

I'm interested in viewing actual converted results though, with back and forth comparison

[gituser123456789000]

As far as the 5 second limitation (or "up to 110 frames")... I'm not sure why they couldn't start dropping the beginning frames out of memory while continuing a longer conversion. At frame 111, drop frame 1 out of memory.. at frame 112, drop frame 2 out of memory.. and continue with a full minutes or hours long conversion. Seems simple enough that that should work, but I'm not a coder