I built this because I was tired of "AI tools" that were just wrappers around expensive APIs with high latency. As a developer who lives in the terminal (Arch/Nushell), I wanted something that felt like a CLI tool and respected my hardware.
The Tech:
GPU Heavy: It uses decord and PyTorch for scene analysis. I’m calculating action density and spectral flux locally to find hooks before hitting an LLM.
Local Audio: I’m using ChatterBox locally for TTS to avoid recurring costs and privacy leaks.
Rendering: Final assembly is offloaded to NVENC.
Looking for Collaborators: I’m currently looking for PRs specifically around:
Intelligent Auto-Zoom: Using YOLO/RT-DETR to follow the action in a 9:16 crop.
Voice Engine Upgrades: Moving toward ChatterBoxTurbo or NVIDIA's latest TTS.
It's fully dockerized, and also has a makefile. Would love some feedback on the pipeline architecture!
Fair point. I used SOTA models for the analysis to prioritize quality, but since the heavy media processing is local, API costs stay negligible (or free).
The architecture is modular, though—you can definitely swap in a local LLM for a fully air-gapped setup.
I don't get this reasoning. You were tired of LLM wrappers, but what is your tool? These two requirements (felt like a CLI and respects your hardware) do not line up.
Still a cool tool though! Although it seems partly AI generated.
Fair points all around. To be transparent: yes, I used an AI coding assistant (Antigravity) to help with the heavy lifting of refactoring the original legacy code and drafting the README. I’m with @rustyhancock on this—I’d rather focus my brainpower on the pipeline logic and hardware integration than on writing boilerplate and Markdown.
However, orchestrating things like decord with CUDA kernels, managing VRAM across parallel processes, and getting audio sync right with local TTS requires a deep understanding of the stack. An LLM can help write a function, but it won't solve the architectural 'glue' needed to make it a reliable CLI tool.
The project is open-source precisely because it’s a work in progress. It needs the 'human touch' for things like the RT-DETR auto-zoom and more nuanced video editing logic. PRs are more than welcome—I'd love to see where the community can push this beyond its current state.
Seems like the post you're replying to has since been edited to clarify that he's referring to the wrappers that rely on third party AI APIs over the internet rather than running locally.
Haha fair enough.The actual internals are basically just one big fight with VRAM. I'm using decord to dump frames straight into GPU memory so the CPU doesn't bottleneck the pipeline. From there, everything—scene detection, hsv transforms, action scoring—is vectorized in torch (mostly fp16 to avoid ooming). I also had to chunk the audio stft/flux math because long files were just eating the card alive. The tts model stays cached as a singleton so it's snappy after the first run, and I'm manually tracking 'Allocated vs Reserved' memory to keep it from choking.
Still plenty of refinement left on the roadmap, but it's a fun weekend project to mess around with.
I watched a video[1] recently that posited the idea of AI slop farms making large, auto-moderated spaces impossible to find meaningful human content in. With the idea that it'll lead to a renaissance for smaller, more personal websites like forums or other niche places to flourish.
I think that sounds a little too convenient and idealistic to be what really happens, but I did find the concept to be a potential positive to what's happening around it. Facebook is already a good portion of the way there, being stuffed with bots consuming stolen or AI content from other bots, with confused elderly people in the middle.
Can I use this for other use cases instead of game videos?
I want to create film-style scenes, cinematic elements, and smooth motion effects.
I’m also thinking of deploying it as a SaaS and using it for video creation features in my app: https://picxstudio.com/
Definitely. The architecture is modular—just swap the LLM prompts for 'cinematic' styles. It's headless and dockerized, so it fits well as a SaaS backend worker
What's the intended use case for this? It seems like you'd create slop videos for social media. I'd love to see more AI use cases that aren't: uninteresting content people would prefer to avoid.
It’s actually designed for your own gameplay—it scans hours long raw session to find the best highlights and clips them into shorts. It's more about automating the tedious editing process for your own content rather than generating "slop" from scratch.
Personal consumption is an interesting angle. I'm starting to think AI content is only desirable to the creator, but no one else wants to see the slop.
big fan of the 'respects my hardware' philosophy. i feel like 90% of ai tools right now are just expensive middleware for openai, so seeing something that actually leverages local compute (and doesn't leak data) is refreshing
The Tech:
Looking for Collaborators: I’m currently looking for PRs specifically around: It's fully dockerized, and also has a makefile. Would love some feedback on the pipeline architecture!This is the first sentence in your features section, so it is not strange if users don't understand if this tool is running locally or not.
Still a cool tool though! Although it seems partly AI generated.
HN is a niche audience but it seems like it's the first question everyone has when opening a repo.
Which is odd because the first question we should have is, does it work.
Personally I can't see myself ever writing the bulk of the README again, life's too short.
However, orchestrating things like decord with CUDA kernels, managing VRAM across parallel processes, and getting audio sync right with local TTS requires a deep understanding of the stack. An LLM can help write a function, but it won't solve the architectural 'glue' needed to make it a reliable CLI tool.
The project is open-source precisely because it’s a work in progress. It needs the 'human touch' for things like the RT-DETR auto-zoom and more nuanced video editing logic. PRs are more than welcome—I'd love to see where the community can push this beyond its current state.
Regardless, we need more tools like this to speed social media towards death.
I think that sounds a little too convenient and idealistic to be what really happens, but I did find the concept to be a potential positive to what's happening around it. Facebook is already a good portion of the way there, being stuffed with bots consuming stolen or AI content from other bots, with confused elderly people in the middle.
[1] https://youtu.be/_QlsGkDvVHU
I did smth similar 4 years ago with YOLO ultralytics.
Back then I used chat messsges spike as one of several variables to detect highs and fails moments. It needed a lot a human validation but was so fun.
Keep going