Changes since DDD Athens

My talk at DrupalCamp Kortrijk was mainly about the changes that have been made after my talk at DDD Athens . The RAG-LLM Translator underwent a major overhaul - The Python library at the core of the system has been retired, a number of small bugs have been fixed, and some new features have been added.

To keep this post concise and meaningful, I won’t stick to the slide deck, but instead I will focus on the main changes. The slide deck is available for download from here .

Here’s a list of changes, and the details are described below:

  • The system menu
  • Token counter / cost estimator
  • Translating into multiple languages in a single run
  • Support for local and various external LLM services
  • Custom embedding model configuration
  • ChromaDB backup & restore (model mismatch guardrail)
  • ChromaDB Admin UI
  • Retirement of gpt-po-translator

The system menu

Previously, when a user wanted to use a certain feature, they had to execute a command at the prompt. The problem was that there were 10+ commands and the usability was poor. I’ve added the system menu so the users know what is available, and it’s much easier to operate.

Token counter / cost estimator

If you are translating a large number of strings, you may wonder how much it will cost you. The token counter / cost estimator gives you an estimate by:

  1. calculating the number of tokens based on the source strings in the .po file, and
  2. estimating the number of tokens the RAG context and prompts may add.

For this feature to work, you need to supply the pricing information of the model you intend to use. Refer to the documentation for further details.

Translating into multiple languages in a single run

Since I built the system for the Japanese community as a proof-of-concept, the system originally supported only one target language, and switching to another language was cumbersome, though not difficult. During the Dev Days in Athens, several people mentioned it would be useful if the system could translate the source strings into multiple languages. Since it was essentially adding some loops, this feature was added shortly after I returned from Athens.

Support for local and various external LLM services

Because I used amazee.ai for the initial phase of the project, which supports OpenAI API standards via LiteLLM, the system only supported OpenAI API standards. Obviously, this meant that people couldn’t use Anthropic models via their API endpoints (and as we all know, most developers use Claude).

Also, there were people who preferred to use their local models, which the system didn’t support.

As a solution, I’ve integrated LiteLLM as an LLM gateway so that local LLMs and most external LLM providers can be utilised.

Custom embedding model configuration

After trying out different text embedding models, I’ve settled on BAAI/bge-large-en-v1.5 for vectorising TM and glossary. But new models are being released and I didn’t want to get stuck with an old model, so I’ve added a feature that streamlines the switching of text embedding models.

ChromaDB backup & restore (with model mismatch guardrail)

I was testing different TMs and glossaries, and often had to wipe and re-ingest, which took time. A backup feature is always useful for a system with a DB, so that has been added.

Since the coordinates of each TM item and glossary term are calculated by a text embedding model, swapping to another model will make the distance calculation of the RAG system invalid. So the restoration feature checks that the text embedding model currently installed matches the one that was used to ingest the data, to ensure the coordinates are valid.

ChromaDB admin UI

Querying ChromaDB is not as straightforward as writing SQL queries (at least for me), so checking the data in the DB was never easy. To make this easier, I’ve added an admin UI for ChromaDB.

Retirement of gpt-po-translator

(This one is probably least exciting for the readers so I’ve placed it at the bottom :p )

gpt-po-translator , the LLM-based .po file translator was what inspired me to build the RAG-LLM Translator. While I worked on the PoC, I used gpt-po-translator as the gettext / batch handler of the RAG-LLM translator, but it soon became apparent that I had to implement a number of workarounds to make my translator work the way I wanted it to. In the end, I replaced it with polib and custom code which made the system free of convoluted workarounds and easier to test.

About the event

Last but not least, DrupalCamp Kortrijk was a great event - the Belgian community organised it amazingly well as always, and I had a great time learning, catching up with friends and making new ones. Many thanks to the volunteers and sponsors for making it happen!

It was also an honour to be featured in an article of the Drop Times .