10000 Stars Later
The Khoj Journey So Far
The journey with Khoj has been an enlightening roller-coaster. Back in 2021, we were dabbling with creating these personal AI search engines - something to sift through our mountains of org-mode notes using a natural language interface. My co-founder Debanjum was turning these AI models into a local document search engine, while I was iterating on making Khoj easy to use, setup for other avid note takers.
Here’s the ride we’ve been on so far:
- We built a local document, image AI search engine for Emacs users. Created the ability to chat with GPT, even about your notes, back in 2021 1. Did decently well on r/OrgMode. Missed being ChatGPT 🫠
- Debanjum started working on Khoj full-time in the summer of ‘22. Our passion project to create an open-source, personal AI called Khoj, reached 300 GitHub stars. We landed a spot in YCombinator for the summer of ‘23.
- Someone hard-launched us on to the front page of HackerNews in July ‘23 but we got roasted for being mere chatgpt wrappers and not being open source enough.
- We did a ShowHN of our new local chat with your docs experience. It got us on the front page of HN, this time with positive vibes.
- On the impulse of closing the accessibility gap, we worked with Meta to integrate our personal AI into WhatsApp.
- In November, I rearchitected Khoj to allow it to scale from a single user, self-hosted experience to a multi-user, cloud service2. We launched the Khoj cloud service!
- Over the next few months we iterated based on user feedback, to turn Khoj into a capable AI agent. It now had the ability to research online, paint images, take on specialized personas and perform tasks autonomously on your behalf.
- Performance was spotty, so we migrated from EC2 to ECS for smoother scaling.
- Some delightful YouTube/Twitter posts made us go trending at #1 on Github. We scrambled to handle the big traffic spike. That earlier ECS migration became a massive lifesaver. Khoj crossed 10K stars. And our UX continued to look like it was straight out of a cartoon.
- We hired our first teammate, Raghav, for the summer, who’s hit the grounding sprinting 🏃🏽♂️➡️
That brings us to today! Cool, so what did we learn?
Building an Open Source AI Company
The dynamics of building an open-source AI company are quite different from closed-source companies. A lot of our early users are hackers and developers. This makes Khoj development more collaborative and transparent with less effort. It means the community can find vulnerabilities, test capabilities, report issues and contribute fixes earlier and faster. It means they can just jump in, get their hands dirty, and solve their problems without our assistance. It means Khoj can get aligned to humans faster and stay aligned with them for longer.
As maintainers of the repository, we need to do a better job of making it super easy for developers to help themselves. Documentation should be tight, complete. If someone asks you a question, answer it, and add the answer to your documentation right away. Your documentation should be really easily searchable (Algolia + Docusaurus is GOAT). You should flag good first issues for getting new contributors started with less effort.
The Discord community has been OP. Most of the people who self-host Khoj make their way there and give ample, active feedback. It’s become a great place to share ideas and build our understanding of what AI will look like. That being said, it’s really hard to scale great support while building your product. Sometimes you also have to say no to support requests, and that can be really hard. Especially when we were working on exclusively self-hosted LLM-application support, there was such a long tail of complex, bespoke user issues that we couldn’t address. That can feel really disappointing.
For engineering founders, your job isn’t just to build cool stuff. You also have to get good at sharing it with people, figuring out how to make something that actually solves an acute user need, and make it look nice. It pays dividends to actually design the UX of what you’re going to build before you build it. And you’ll thank yourself for creating specs before making architecture decisions. It makes it much easier to rubber duck your thought process and share it with your team. Don’t write specs for single API endpoints.
Your team should be able to manage communication in a Slack channel or a WhatsApp group. You don’t need a massive Notion or Jira dashboard when it’s just two of you and an intern. Stay lean, keep your processes lightweight and efficient. Do weekly and monthly planning to stay focused.
The last year has been really eventful, full of learnings and adaptations. Biggest lesson is to keep your ear to the ground, really listen to what pain points people are having, iterate on it quickly and cut out the noise. We’re not perfect, but we’re learning.
Convictions about AI
Through all this, we’ve also picked on a few pieces of conviction about AI:
- AI will fundamentally shift how we understand and access information. We need to carefully design our AI interfaces to ensure this shift improves our capabilities.
- Open-source AI is much faster and easier to keep aligned to individual human interests. We need our personal AI to be aligned to us and minimize chances of misalignment wherever possible. Even if the creators change, people should be able to take ownership of the service and ensure service/user alignment.
- Communication will get easier for people across languages, personas and mediums. But we need to ensure our communication channels do not get flooded with reality warping noise generated by AIs and humans.
How we construct these new machines over these next few years will decide if AI improves the human condition. That is our north star. That is what we continue to innovate towards.