This is what I wanted to help with duplicate code detection, which is now a real problem with agentic programming since the agent tends to duplicate helper functions a lot in bigger codebases.
I ended up writing my own tool [1] that uses vector search, which works but can be unusably slow in large codebases.
ooh I'll give yours a shot - thanks for calling my project out on your readme!
I was fortunate to have a contributor push me to consider optimizations a few months back, and it definitely helped improve the speed/memory metrics.
I see the CLI knobs are somewhat similar ('min lines', 'thresholds'), however your 'search' is fascinating. I could see myself wondering "how much error handling is duplicated for HTTP responses?" and your tool would give me a start at an answer. Nice!
I keep meaning to write a post for how this has worked for my projects, I think its been helpful.
For active projects I setup a weekly-ish 'code audit' agent: it runs treepeat to find opportunities to refactor and bumps up enforced coverage numbers. Over time for some projects I lower the '--similarity' threshold to more broadly find similar structured code (potentially more ambitious refactoring)
I ended up writing my own tool [1] that uses vector search, which works but can be unusably slow in large codebases.
I will give this a shot.
[1]: https://github.com/ninjaxtools/slopdex
I was fortunate to have a contributor push me to consider optimizations a few months back, and it definitely helped improve the speed/memory metrics.
I see the CLI knobs are somewhat similar ('min lines', 'thresholds'), however your 'search' is fascinating. I could see myself wondering "how much error handling is duplicated for HTTP responses?" and your tool would give me a start at an answer. Nice!
For active projects I setup a weekly-ish 'code audit' agent: it runs treepeat to find opportunities to refactor and bumps up enforced coverage numbers. Over time for some projects I lower the '--similarity' threshold to more broadly find similar structured code (potentially more ambitious refactoring)