Khoj as a Personal AI Accountant

ai accountant plaintext accounting workflows
Debanjum Singh Solanky
Debanjum Singh Solanky

I’ve been maintaining my personal finances using a plaintext accounting tool called Beancount, for the last decade. These days Khoj helps me keep my accounts synced and maintain an up-to-date view into all my finances.

Bank Statement to my Ledger

I ask Khoj to convert all my transactions from a bank statement PDF into the appropriate plaintext format. This makes it easy to keep my ledger synced with reality.

Khoj intelligently extracts the transactions from the PDF statement, categorizes expenses into appropriate accounts and formats the information into Beancount format.

The flexibility offered by using AI instead of traditional programming is the best thing about this approach. LLMs are super versatile translators. So this approach can be easily adapted to work with other Bank accounts, output formats and changes to the PDF statements. All it takes to “reprogram” your AI accountant is natural language.

Process

  1. Download monthly account statements from my bank(s) as PDF
  2. Start a new conversation with Khoj and add a bank statement to the conversation
  3. Ask Khoj to summarize the bank statement with something like the following message. Update this query to fit your ledger and bank statement format.
/summarize Extract Transaction Date, Merchant Name and Amount from all transactions in my shared credit card statement. Convert it to a Beancount format. Always assume account extracted from is BestCreditCard shown below. Choose Expense account from "Expenses:Food:Dining", "Expenses:Food:Groceries", "Expenses:Transport:Cab", "Expenses:Hardware" and "Expenses:Miscellaneous". Use the miscellaneous account if do not know which expense category to use. 

Example: `04/01, UBER *TRIP HELP.UBER.COM, 4.20` extracted from PDF to Beancount format:
```
2024-04-01 * "Uber" "Cab ride" #generated
  Expenses:Transport:Cab
  Liabilities:CreditCard:BestCreditCard  -4.20 USD
```

Give the transactions in a single beancount codeblock
  1. Do a quick scan to compare Khoj extracted transactions to the transactions in the PDF statement.
  2. Copy the extracted transactions, paste it into your ledger and edit any transactions if required

Transaction SMS to my Ledger

Warning: This setup is for advanced users only. It requires an Android phone with Tasker, Termux and Termux:Tasker setup. A Khoj API key and ability to write single file Python (that interacts with Khoj via API).

Khoj automatically adds all my UPI transactions into my ledger. Using the information in the UPI transaction text message:

  • Khoj converts INR to USD based on today’s exchange rate
  • Categorizes the transaction to the most appropriate expense account based on the payee information in the message
  • Compiles the above information into a transaction in the appropriate text format

Implementation

  1. Setup a Tasker profile to trigger on every transaction SMS from your UPI provider. Have it call a simple Python script via Termux.
  2. The Python script interacts with Khoj over the Cloud API to do the following:
    1. Ask Khoj to convert INR to USD. Khoj looks up today’s exchange rate online for this.
    2. Ask Khoj to categorize the new transaction into the provided Expense accounts. E.g Expenses:Food:Dining
    3. Append the Khoj generated transaction to my local Ledger file

Note: I use Syncthing to keep my Ledger synced across my devices

Automations

You can create automations with Khoj to help get reminders, notifications for your finances. For example:

  • Ask Khoj to remind you to share your bank statements and sync your ledger
  • Ask Khoj to remind you to do taxes before tax day
  • Ask Khoj to notify you if the stock market drops by >5% in a single day

Future Improvements

Khoj is getting new capabilities. Here are some future improvements I’m looking forward to that will make my workflows even more seamless:

  • Create Your Personal Accountant Agent
    Next week, Khoj agents will have gained new capabilities. You’ll be able to:
    • Create private agents. E.g Personal Financial Advisor.
    • Tune agent personality for personal use-cases. E.g Chase to Ledger accountant.
    • Give them access to specific documents. E.g ledger, bank statements.
    • Restrict tool they can access. E.g no online search.
  • Ability to Reply to Khoj Automation Emails
    It’d be super convenient if you could reply and attach your Bank statements to Khoj’s email reminding you to sync your ledger. This would make it even easier to keep your ledger synced.

Conclusion

Access to advanced AI agents means everyone can program using natural language. But we’re all still discovering the best way to use these powerful tools.

My setup will not apply to most folks. And that is fine. Khoj helps me with stuff that is unique to me.

This post is meant to inspire you to create workflows with Khoj that fit your unique needs like a glove. And then share your tailored workflows so we can all learn how to better use these tools in our various endeavors.