Marlin: Personal 500k-Domain Search Engine Built With Local LLM

TL;DR. A developer created a 500,000-domain personal search engine called Marlin over a weekend using a local language model. - The project focused on cataloging creative homepages and zines, avoiding corporate SEO-heavy content. - The system processed website metadata with an LLM and stored it in a Postgres database for fuzzy searching. - The entire setup cost approximately 0, including an overnight GPU rental for initial processing.

• Developer built a personal search engine named Marlin for creative content.
• The search engine uses a small local language model to process website homepages.
• Project took a weekend and cost around 0, utilizing a rented GPU.
• Marlin filters out corporate sites, focusing on independent makers' content.

Back to QLANKR News