I came across this last week when I was looking to resurrect an old scrapy based project. I’m not sure if it’s worth rewriting what I have already (especially because I’m the only one on the team who actively enjoys JS/TS), but I’d definitely give it a shot for something greenfield.
We were discussing private search engines in another thread, this seems like a cool piece of that puzzle
Do you have a link handy? I’m interested.
https://lobste.rs/s/dj1zzl/stract_web_search_done_right was that discussion.
I think https://github.com/paradedb/paradedb would be a good backend as they have proper search support, columnar storage and S3 support
I came across this last week when I was looking to resurrect an old
scrapybased project. I’m not sure if it’s worth rewriting what I have already (especially because I’m the only one on the team who actively enjoys JS/TS), but I’d definitely give it a shot for something greenfield.Has anyone worked with both?