Springing forward in our web stack
Our web stack at Khoj has been quite unique. We have a Python web server using FastAPI as our app layer and Django for ORM and admin pages. While this served us fine initially, it’s led to some performance issues that keep me up at night and deserve their own discussion.
Our front-end journey is equally interesting. We recently launched a brand new UX (we call it Spring UX), migrating our front-end code base in the process.
For three years, we served pages using Jinja templates with FastAPI. This approach allowed for partial server-side rendering but meant building web components from scratch. Yes, we were using plain HTML, CSS, and JavaScript even after reaching 10,000 users! We chose this path of under-engineering for several reasons:
- Lightweight static HTML pages
- Simplified development without cross-OS compatibility issues
- Abundant solutions to common problems, making AI assistance more effective
- Avoiding time-consuming migration to modern frameworks while validating our product
This strategy worked well initially, but as we grew, limitations became apparent. We faced development bottlenecks due to writing core code from scratch and struggled with code duplication. As we prepared for a major redesign, it became clear that our front-end needed an overhaul too.
We needed a graceful migration path that minimized server-side changes. After considering options like Svelte and HTML Components, we settled on Next.js with static site generation. This choice allows us to:
- Leverage popular component libraries
- Migrate without a complete overhaul
- Minimize new technology risks
- Potentially move our front-end entirely to a CDN in the future
We adopted the startup starter pack ™️, aka Next.js, Tailwind, and Shadcn.
An additional consideration was our Python package on PyPI. We needed the new front-end pages to work within this package. Our solution involves building files in our GitHub action, packaging them into a distribution directory, and serving them from a static directory.
Since then, we’ve moved from a cartoon-like UX to a more colorful, modern interface we call “spring” – symbolizing both our visual refresh and a leap forward in user experience.
This journey highlights the evolution of our tech stack, balancing simplicity with the need for scalability and “modern practices”. Eventually, you do actually have to remove the stilts on your codebase as your product grows and user needs evolve.
| Before | After |
|---|---|
 |  |