Laravel toolkit for machine learning

Laravel toolkit for machine learning

Why do the Python guys have to have all the fun?

I am working on a toolkit to help Laravel developers harness the power of machine learning. The goal is to make building RAG-type applications much easier. In the Python world, there is Langchain, and there are PHP "wrappers" for this toolkit (Love Langchain btw!) but the biggest problem with wrappers, is that Langchain is a vast toolset and can become confusing for developers not familiar with machine learning.

I have taken the simple approach, to favor convention over configuration!

Why machine learning in PHP?

PHP is not designed for machine learning, however, in the past year or so there's been a monumental shift towards building RAG applications. This then allows developers to build custom machine-learning applications without needing to train their own models.

You simply use a third-party API such as OpenAI or run your open-source LLM such as Llama3. In either case, you can get quite a bit of mileage from PHP by just making rest calls to the backend LLM API.

I won't go into too much detail about RAG, since this is beyond the scope of this article but if you want to understand more about RAG, please check out my other articles on this blog including "How to build a Chatbot with Faiss".

Just to summarize, RAG enables developers to provide custom business data as context for the LLMs.

Providing a custom dataset to the model has quite a few benefits:

  • Scoped responses: For example, If you ask ChatGPT: "Which is the best car insurance cover for my car?" the LLM will give you a generic answer. In your custom RAG though, you can feed the LLM user-specific data and tailor the response based on what your company offers and the user's unique profile data.

  • Realtime: In addition to just static training data, you can also feed the LLM data from APIs in real-time.

  • Prevent or minimize hallucination: Since the model is scoped to your data, it's not going to give the user some random responses, you can force the LLM to either pull the data from your dataset or simply return "I don't know".

Coming back to PHP, PHP is still one of the most popular web development languages and is used throughout the web for both small to large-scale websites, thus it makes sense to build a library to help this user base gain access to powerful LLMs and RAG systems.

Ragable - The PHP Machine learning toolkit

Even in the Python world with powerful libraries like Langchain, it can become a bit confusing, how to build multi-conversation workflows. The technology is still very new and still rapidly evolving.

Being an early adopter myself, I have had an unfair advantage of building my skill set while the technology was in early infancy, thus, I have been learning along the way and have gradually used most of the components you would need to build most common RAG type applications.

Ragable is my attempt to simplify and bring machine learning to PHP. With Ragable you will be able to build complex agents using a declarative PHP syntax, furthermore, I have hand-picked the tooling that makes sense for most applications. This will save you hours, if not days of research and fiddling time.

Here's a sneak peak:

use Ragable\Adapters\QdrantAdapter;
use Ragable\Adapters\OpenAiAdapter;
use Ragable\Types\VectorDocument;
use Ragable\Agent;
use Ragable\AgentTask;

$llm = OpenAiAdapter();
$qdrant = new QdrantAdapter("documents", $llm);

$docs = [];
foreach($pds as $pdf) {
   $docs[] = VectorDocument($pdf->text);
}
$qdrant->addDocuments($docs);

$agent = new Agent($llm);
$task = new AgentTask(
    "City info guide",
    "Execute this task if the user asks about cities."
);
$task->askLLM = false;

$task->alwaysExecute = false;

$task->executeOrder = 0;

$task->executable = function($question, $params) use($qdrant) {
   $docs = $qdrant->findSimilar($question, 1);
   if (!isset($docs[0])) {
     return $docs[0]->text;
   }

   return "The capital city is Johannesburg";
};

$agent->registerTask($task);

$question = "What is the capital city of South Africa?";
$finalResponse = $agent->run($question);

Let's break down what is happening here:

  • First, we ingest text data and convert it to vector embeddings.

  • These embeddings are stored in Qdrant.

  • We set up an Agent, which takes an LLM instance. In this case, we using OpenAI but you can use any other supported LLM.

  • Next, we add tasks to the agent. Based on the user's question, the agent will determine which is the best task to execute and execute that task.

  • Within our task, we query Qdrant to get the most similar document and return the text from this document. Since "askLLM" is false, the Agent will not post this output to the LLM but rather just return the response of this function back to the user.

  • Finally, we run the agent, this will query the LLM, run tasks, and eventually return the final response to our user.

What is Qdrant?

To build a RAG application, you are going to need a vector database, usually, this is a pain to figure out which provider to use, how to connect to the DB, how to use their API, and so forth. From personal experience, indexing hundreds of thousands of documents, I found Qdrant to be a pretty solid option.

To set up Qdrant, you simply need to run the docker image and the "QdrantAdapter" class will take care of the rest in terms of vectorizing and querying Qdrant.

To run the docker image:

docker run -p 6333:6333 -p 6334:6334 \
    -v $(pwd)/qdrant_storage:/qdrant/storage:z \
    qdrant/qdrant
$llm = OpenAiAdapter();
$qdrant = new QdrantAdapter("documents", $llm);

$docs = [];
foreach($pds as $pdf) {
   $docs[] = VectorDocument($pdf->text);
}
$qdrant->addDocuments($docs);

In the above code example, we have parsed some PDF documents (your data can come from anywhere including the DB) and inserted them into Qdrant.

Behind the scenes, the "QdrantAdapter" will convert the PDF text into vector embeddings, and save those vectors in Qdrant. It invokes the OpenAI embedding API under the hood (You will notice we pass in $llm, this tells Qdrant how to query OpenAI or whatever LLM you use).

LLMs have a limitation on their context window, plus, you pay for every single token sent to the API. The purpose of using Qdrant is to filter out your data so that only the most relevant documents/text is returned.

Thus, you can then post a smaller amount of context data which the LLM will analyze and look for information that it needs to best answer the user's question.

Under the hood, Qdrant is using an "ANN" (Approximate nearest neighbor) Algorithm to find documents that have similar contextual meanings based on the user's question. This is why we store vector embeddings instead of raw text.

Vector embeddings are essentially floating point numbers. This allows Qdrant to apply a cosine similarity math calculation, to determine the contextual meaning of the text and how close in proximity they are to each other.

What are Agents and Agent Tasks?

You will notice in the code below we instantiate an "Agent" class. Usually, when you interact with an LLM, you prompt the LLM with text and the LLM returns a text response.

$agent = new Agent($llm);
$task = new AgentTask(
    "showCart",
    "Execute this task if the user asks about thier shopping cart."
);
$task->askLLM = false;
$task->alwaysExecute = false;
$task->executeOrder = 0;
$task->executable = function($question, $params) {
   return MyCart::getFormattedResponse();
};

This is not very useful for more complex applications, a user might ask: "Please remind me what items are in my shopping cart?"

When you create an AgentTask , the second argument passed is used by the Agent to figure out when to execute this task.

The LLM is not going to understand this question, and give the user a generic response or "I don't know", by using an agent, you can dynamically invoke functions and feed the LLM different context data that is relevant to the user's question.

These custom functions (Agent Task) can then perform any number of operations, including searching a local DB, doing a vector search, or querying an API. Depending on your "askLLM" setting, the Agent can then return the output of the function as a reply to the user or it can forward the function's response back to the LLM for the LLM to format and apply reasoning and return a better answer to the user.

In the example above, the function we invoked is "showCart", but what if the user asks for the shop's contact details? Executing "showCart" would be useless.

In this scenario, we can register a second task "shopDetails", therefore, the agent can now smartly pick "shopDetails" whenever the user asks about the shop's contact information.

Using this technique you can build quite complex mult-conversation applications. Example: an ecommerce ordering system, based on the users input, you can create similar flows to "Add to cart" , "view product" etc... instead of the user clicking buttons, you simply route to the relevant function based on natural language.

Agent tasks are very flexible and offer the following customizations:

  • Regular PHP functions: You can pass any custom data to the task including a user object. Since functions are not "OpenAI functions", the function schema and any data you pass to the function remain private and are never sent to the API. Only the return value is sent and the response of the LLM is provided to the function as an argument.

  • Can be executed in a particular order: You can easily pipe the response from one function to the next, and control in which order functions are executed.

  • Execute always: By default, an Agent Task is only executed if it is relevant to the user's question, and the Agent can only execute one task, however, if you turn on "alwaysExecute", then the task will always execute. This is useful if you want to log the agent data or format the response in a particular way before returning the final answer.

  • Ask the LLM: The agent by default will take the Agent Task response and send it to the LLM for further reasoning, however, you can turn this off and return the response from the function as is.

  • Easy memory management: The Agent automatically manages memory, so that when you build multi-turn conversations, the LLM is aware of any prior information that was exchanged between the Agent and the user.

If you like this idea, and want to be notified when the tool is launched. Please subscribe to my newsletter here.