The problem is I don't like XAML, the XML-based markup language used by WPF (Windows Presentation Foundation). WPF is basically a UI framework for building Windows desktop applications, and apparently, the Dotnet crowd prefers this over WinForms these days 🤷.

Luckily! I discovered wails, an interesting project that essentially uses a WebView to build Windows Forms UIs in CSS and JavaScript. Additionally, you can connect those UI elements to a Go backend and even access native Windows APIs, similar to how you would have a code-behind button click handler in C# WinForms.

It's also no secret that I'm a big fan of Golang. It's fast, it's clean, and I enjoy writing in Go all the time. I've built everything from scrapers to a full-on vector database in Go, so why not take a whack at building a desktop app too 🤷.

Anyway, I started experimenting with building a simple parental control app. The web can be a dangerous place, and I have an 8-year-old who loves binge-watching cartoons on my laptop after sucking the life out of every battery-operated mobile device in the house 😆.

So, as you can imagine, I’d like to block all the bad junk on the internet or at least filter out the obvious bad sites and inappropriate content. This article is merely an experimental log of my journey and is not meant to be used in an actual production app. You probably would be better off subscribing to one of the existing parental control apps on the market.

Which approach should we use?

TLDR: We'll use Windows API's to get a list of active windows, then scan their titles, and finally close running programs that contain inappropriate content.

There are many approaches to building a solid parental control app; as mentioned, we're just experimenting for now and not looking to build anything too overly complicated. Some options we have:

• A VPN service. With a VPN-based approach, all network traffic is routed through your app. This allows your app to inspect the connections being made and potentially analyze the content being accessed. In theory, this means you could decrypt HTTPS traffic and inspect full web pages, including the document body, images, videos, and other resources. As you can imagine, this is a pretty solid approach and can help you build a fairly robust system.

• A DNS service. While the VPN approach is the most comprehensive solution, decrypting HTTPS traffic comes with serious security risks. If your VPN implementation is compromised, it could effectively create a man-in-the-middle (MITM) attack, allowing someone to intercept sensitive user data. Instead, we could just build a lightweight DNS service that allows us to read the relevant hostnames (mywebsite.com). This way we can safely monitor incoming traffic and block bad websites without actually decrypting anything.

• ETC Hosts. Another simple approach that works surprisingly well. Instead of building and managing a DNS service (which is slightly more complex), we can simply modify a small system file called hosts, located at:

  C:\Windows\System32\drivers\etc\hosts

  As a web developer, I use this file all the time to map local domains during development. The same mechanism can also be used to block websites. By mapping bad domains to 127.0.0.1 any request to those domains will just be redirected back to the local machine, effectively preventing the site from loading. I actually got this working nicely in the C# version.

• Window titles. Even simpler, and this is the approach we'll use in our Golang program. With this approach, you basically get a list of all running windows and inspect their title. Many websites include SEO keywords in the page title, which often contains the bad words we're looking for. By scanning these titles for certain keywords, we can easily detect when inappropriate content is being viewed and respond accordingly.

Setting up Wails

Installing wails is a breeze, just like any other Go library:

go install github.com/wailsapp/wails/v2/cmd/wails@latest
wails init -n parental-control-app -t vanilla-ts

The second line will set up a Skeleton project for you, which would look like this:

├── README.md
├── app.go
├── go.mod
├── go.sum
├── main.go
├── lib
│   └── program_scanner.go
├── myproject.exe
├── parental-control-app
│   ├── README.md
│   ├── app.go
│   ├── go.mod
│   ├── go.sum
│   ├── main.go
│   └── wails.json
└── wails.json

I excluded the "frontend" folder from the tree structure. This is where your "UI" lives:

├── index.html
├── package-lock.json
├── package.json
├── package.json.md5
├── src
│   ├── app.css
│   ├── assets
│   │   ├── fonts
│   │   │   ├── OFL.txt
│   │   │   └── nunito-v16-latin-regular.woff2
│   │   └── images
│   │       └── logo-universal.png
│   ├── main.ts
│   ├── style.css
│   └── vite-env.d.ts
├── tsconfig.json
└── wailsjs
    ├── go
    │   └── main
    │       ├── App.d.ts
    │       └── App.js
    └── runtime
        ├── package.json
        ├── runtime.d.ts
        └── runtime.js

I chose the vanilla template to keep things simple, but Wails does support React, Vue, and a bunch of other JS frameworks too, so pick your poison accordingly ✊

When you open up the main.go file, you'll see something like this:

package main

import (
	"embed"

	"github.com/wailsapp/wails/v2"
	"github.com/wailsapp/wails/v2/pkg/options"
	"github.com/wailsapp/wails/v2/pkg/options/assetserver"
)

//go:embed all:frontend/dist
var assets embed.FS

func main() {
	// Create an instance of the app structure
	app := NewApp()

	err := wails.Run(&options.App{
		Title:  "My App",
		Width:  1024,
		Height: 768,
		AssetServer: &assetserver.Options{
			Assets: assets,
		},
		BackgroundColour: &options.RGBA{R: 27, G: 38, B: 54, A: 1},
		OnStartup:        app.startup,
		Bind: []interface{}{
			app,
		},
	})

	if err != nil {
		println("Error:", err.Error())
	}
}

Fairly straightforward, this is basically setting up a standard Windows with a title, a height background, and so forth.

The most interesting thing in the boilerplate code is this part:

		Bind: []interface{}{
			app,
		},

This is where the JavaScript magic happens. We bind our app struct, so its methods become callable from the frontend JS code.

Let's peek inside app.go:

package main

import (
	"context"
	"fmt"
	"myproject/models"
)

// App struct
type App struct {
	ctx context.Context
}

// NewApp creates a new App application struct
func NewApp() *App {
	return &App{}
}

// startup is called when the app starts. The context is saved
// so we can call the runtime methods
func (a *App) startup(ctx context.Context) {
	a.ctx = ctx
}

// Greet returns a greeting for the given name
func (a *App) Greet(name string) string {
	return fmt.Sprintf("Hello %s, It's show time!", name)
}

Nothing fancy, but pay close attention to the "Greet" method. This is a function that is now exposed to the frontend, and you can call it from anywhere in your JS/TypeScript code just like any other function.

Next, look inside frontend/src/main.ts:

window.greet = function () {
    // Get name
    let name = nameElement!.value;

    // Check if the input is empty
    if (name === "") return;

    // Call App.Greet(name)
    try {
        Greet(name)
            .then((result) => {
                // Update result with data back from App.Greet()
                resultElement!.innerText = result;
            })
            .catch((err) => {
                console.error(err);
            });
    } catch (err) {
        console.error(err);
    }
};

Notice the "Greet" in the try catch! We are effectively calling the Greet method on our App struct 🎉, how powerful right?

Okay, but where’s the native Windows stuff?

We managed to create a UI in TypeScript and then fire a function in Golang, but you could just launch a headless browser and use WASM to do the same thing, right? So, what's the point?

Yee of little faith 😏!

Since we can run Go code and Go has bindings for Win32 APIs, we can effectively access almost any native functionality we need.

If you don't know what Win32 is, basically it's a standard set of APIs that the Windows OS heavily relies on to manage low-level systems like the UI, networking, security, and so forth. Whether you've used XP, Vista, 7, 10, or 11, you've probably noticed that decade-old programs still run perfectly. This is largely thanks to Win32, enabling Microsoft's exceptional backward compatibility.

Essentially, it's a collection of DLLs, or Dynamic Link Libraries. If you think of this in terms of Golang, a DLL is similar to a package containing code that you can import and use. The key difference is that DLLs are precompiled and shared across programs at runtime, rather than being compiled into your binary like Go packages, but the concept is similar.

Golang provides a way to interact with Win32 via this package:

golang.org/x/sys/windows

And to enable access to Win32 API's, it's as easy as follows:

user32 = windows.NewLazySystemDLL("user32.dll")

In this case, we import user32.dll, which is a DLL that gives us access to the general Windows UI and windowing system, this is exactly what we'll need to access running programs and essential information about those programs to decide whether it's bad content or not.

NewLazySystemDLL allows Go to load the DLL lazily, meaning the DLL and its functions are only loaded when they are actually needed instead of when the program starts.

Moving along, lets setup references in Go to the functions we need to access:

var (
	user32                       = windows.NewLazySystemDLL("user32.dll")
	procEnumWindows              = user32.NewProc("EnumWindows")
	procIsWindowVisible          = user32.NewProc("IsWindowVisible")
	procGetWindowTextLengthW     = user32.NewProc("GetWindowTextLengthW")
	procGetWindowTextW           = user32.NewProc("GetWindowTextW")
	procGetWindowThreadProcessId = user32.NewProc("GetWindowThreadProcessId")
	procPostMessageW             = user32.NewProc("PostMessageW")
)

In addition to importing the user32.dll DLL, we’re also creating pointers to several Win32 functions. These act as a bridge between Go and the native Windows API. Once defined, we can call these functions and pass arguments to them almost as if they were normal Go functions.

• EnumWindows - Will give us a list of running programs.

• IsWindowVisible - Checks whether a window is visible by Windows, even if it’s minimized or behind other windows.

• GetWindowTextLengthW - Basically string.length of a Windows title bar, this allows us to allocate a buffer of the correct size before reading the text.

• GetWindowTextW - As you probably guessed by now, this is the actual title bar text.

• GetWindowThreadProcessId - We'll use this to get the program's process ID so that we can terminate the program if we detect bad content.

A working prototype

Finally, here's the full Go code for V1 of our parent control app:

//go:build windows

package models

import (
	"strings"
	"syscall"
	"unsafe"

	"golang.org/x/sys/windows"
)

var (
	user32                   = windows.NewLazySystemDLL("user32.dll")
	procEnumWindows          = user32.NewProc("EnumWindows")
	procIsWindowVisible      = user32.NewProc("IsWindowVisible")
	procGetWindowTextLengthW = user32.NewProc("GetWindowTextLengthW")
	procGetWindowTextW       = user32.NewProc("GetWindowTextW")
	procPostMessageW         = user32.NewProc("PostMessageW")
)

func ScanWindowTitles(hwnd uintptr, lparam uintptr) uintptr {
	vis, _, _ := procIsWindowVisible.Call(hwnd)

	if vis == 0 {
		return 1
	}

	tlen, _, _ := procGetWindowTextLengthW.Call(hwnd)

	if tlen == 0 {
		return 1
	}

	buf := make([]uint16, tlen+1)
	procGetWindowTextW.Call(hwnd, uintptr(unsafe.Pointer(&buf[0])), tlen+1)
	title := windows.UTF16ToString(buf)
	if title == "" {
		return 1
	}

	if strings.Contains(title, "lotto") {
		procPostMessageW.Call(hwnd, 0x0010, 0, 0)
	}

	return 1

}

func RunBadAppChecker() error {

	windowScannerCallback := syscall.NewCallback(ScanWindowTitles)
	result, _, err := procEnumWindows.Call(windowScannerCallback, 0)
	if result == 0 {
		if err != nil && err != syscall.Errno(0) {
			return err
		}
		return syscall.EINVAL
	}

	return nil
}

The naming convention is a bit weird; it comes from the C-style approach of Win32. Here's a breakdown block-by-block of what's going on.

The logic behind this code:

• windowScannerCallback := syscall.NewCallback(ScanWindowTitles) ~ converts a Go function into a Windows-compatible callback function, essentially a pointer, so it can be passed to Win32 APIs correctly. Additionally, this prevents Golang's garbage collection from getting "confused". Since we sort of tunnelling through to native libraries, the Garbage collector isn't fully aware of its state, thus it might try to collect or move this around in memory when it shouldn't.

• result, _, err := procEnumWindows.Call(windowScannerCallback, 0) ~ We basically loop through all open windows and run this callback on each one. Similar to an array map function. The second argument 0 (of type uintptr ) is just an additional state argument we can pass to the function; in this case, we are not using it, so we set it to 0.

💡 The uintptr type can hold a large integer representing a memory address. In Go terms, when you pass &someVariable to get a pointer, you're getting a memory address - uintptr is just that address stored as a plain integer.

Looking inside ScanWindowTitles We first check if the current window is visible:

	vis, _, _ := procIsWindowVisible.Call(hwnd)

	if vis == 0 {
		return 1
	}

Next, we check if the window has a title:

	tlen, _, _ := procGetWindowTextLengthW.Call(hwnd)

	if tlen == 0 {
		return 1
	}

💡 hwnd ~ is just a Windows convention for an ID. Think of it like HTML: you give a <div> an ID to reference it in CSS/JS. Similarly, hwnd is an ID that lets you reference and interact with a specific window through Windows API calls.

At this point, we now have a window that's a) visible and b) has a title. So, this means we can scan it for bad words.

PS: Much of what you've seen thus far in this code is ceremonial C/C++, since Windows is built on these languages for historical and performance reasons. As Go developers, we're very privileged to have a high-performance language that's also clean and abstract enough without compromising too much on performance.

Sure, C/C++ will outperform Go in nearly all cases, but for modern computing, especially for web developers, that kind of raw performance is often overkill. Nonetheless, since we are interfacing with Win32, we've just got to swallow that hard pill of C/C++ style programming!

Anyway, getting back to our window title scanner, we have this weird-looking block of code:

	buf := make([]uint16, tlen+1)
	procGetWindowTextW.Call(hwnd, uintptr(unsafe.Pointer(&buf[0])), tlen+1)
	title := windows.UTF16ToString(buf)
	if title == "" {
		return 1
	}

We create a buffer of type uint16 - essentially an array with a fixed size of elements that will hold the characters making up the window title. Windows uses UTF-16 encoding, which encodes each character as one or two 16-bit values.

The length of this buffer will be n+1 where n is the number of the characters in the title plus one **null terminator (**like an invisible character that has no relevance to the actual string besides being a marker). The null terminator is a bit weird for us Go developers, but this is how C-style strings work; they need a way to know where the string ends.

procGetWindowTextW.Call(hwnd, uintptr(unsafe.Pointer(&buf[0])), tlen+1)
if title == "" {
  return 1
}

We're essentially extracting the window title and writing it to the buffer we created earlier - similar to an I/O file write operation. We return one if nothing came through; this is a Win32 standard way of returning status codes. In this case, we return 1, which means just skip ahead to the next window.

Now, we finally get to the checking for bad words part. I made this super simple, but in reality, you'd probably want to check if the value is in a list or do a DB lookup of some sort.

if strings.Contains(title, "lotto") {
   procPostMessageW.Call(hwnd, 0x0010, 0, 0)
}

// close window function

0x0010 ~ is a hexadecimal message code (Constant in Win32 API: WM_CLOSE) that tells Windows we want to close that window.

PostMessageW is sort of like a postman; we can send messages to the underlying WIN32 API. This API does not wait for the response, so it's essentially a message queue and asynchronous. Since older systems had memory constraints, old-school C/C++ often used constant hexadecimal or integer values to represent actions.

The C# approach

While this article is not really about C#, I did also experiment with the Dotnet ecosystem to see how far I can take this. It was very easy, actually 🤔 Although to be honest, I got a lot of help from Claude Code.

❗ Beginner developers be careful! I don't suggest using AI to learn initially. I strongly suggest reading books first, and spending 3-6 months coding on your own. I use AI to learn sometimes and generate code, but I have a lot of programming experience. I may not fully understand C#'s syntax and API's, however, I understand the logic behind what is happening and can easily steer the AI in the right direction when it goes off-track.

I didn't cover this aspect in the Go code yet, but here we go:

var hosts = File.ReadAllLines(Config.HostsPath).ToList();

hosts.RemoveAll(line => line.Contains(Config.HostsMarker));
hosts.Add($"\n{Config.HostsMarker} - START");

foreach (var url in Config.BlockedUrls) {
 hosts.Add($"127.0.0.1 {url} {Config.HostsMarker}");
 hosts.Add($"{Config.HostsMarker} - END");
 File.WriteAllLines(Config.HostsPath, hosts);
}

In our C# version, we open the hosts file and write a list of bad hostnames to the file, each entry points to 127.0.0.1 (localhost). This ensures that whenever you try to visit any of these domains in your browser, the request gets redirected to your own machine instead of the actual website.

Since nothing is typically running on port 80 locally, the browser will either show an error, display nothing, or render whatever service you happen to have running on that port.

Before writing, we also search all lines that contain our custom marker and delete them. The marker is simply a string like: # BAD DOMAINS that we append to the end of each line, so that we can keep track of any changes we make to the file. This is important because the file can be edited by the OS or users at any time, and we want to ensure that we don't interfere with any lines not managed by the program.

For the title scanner, we can implement it in this way:

foreach (var proc in Process.GetProcesses())
{
    try
    {
        var title = proc.MainWindowTitle;

        if (CheckIfBannedKeyword(title, out string? matchedKeyword))
        {
            if (matchedKeyword != null)
                BlockContent(proc.MainWindowHandle, proc.ProcessName, title, "Keyword", matchedKeyword);
        }

    }
    catch { }
}

static bool CheckIfBannedKeyword(string text, out string? matchedKeyword)
{
    matchedKeyword = null;
    if (string.IsNullOrEmpty(text)) return false;
    var lower = text.ToLower();

    foreach (var term in Config.BlockedTerms)
    {
        if (lower.Contains(term))
        {
            matchedKeyword = term;
            return true;
        }
    }

    return false;
}

static void BlockContent(IntPtr windowHandle, string browserName, string windowTitle, string triggerType, string triggerValue)
{
    if (windowHandle != IntPtr.Zero)
    {
        db.LogViolation(triggerType, triggerValue, browserName, windowTitle);

        SetForegroundWindow(windowHandle);
        Thread.Sleep(100);

        keybd_event(VK_CONTROL, 0, 0, UIntPtr.Zero);
        keybd_event(VK_W, 0, 0, UIntPtr.Zero);
        keybd_event(VK_W, 0, KEYEVENTF_KEYUP, UIntPtr.Zero);
        keybd_event(VK_CONTROL, 0, KEYEVENTF_KEYUP, UIntPtr.Zero);
    }
}

We:

• First, get a list of running processes.

• Next, get the Window title and check if it contains any strings that match a word in our banned keywords list.

• Finally shutdown that process.

In most tech projects, the first 80% is relatively straightforward: CRUD operations, database schema design, and boilerplate code. The real challenge lies in the remaining 20%, the core business logic, and the actual problem being solved. That part demands deep domain knowledge, experience, and sustained effort ~ things LLMs cannot simply replicate.

LLMs have gotten bigger, more resource-intensive, and are eating up GPUs like there’s no tomorrow! They’ve swallowed up the entire internet, but still nowhere close to replacing human beings. AGI is not coming; it’s a get-rich-quick pipe dream that was concocted by all these billionaire tech CEOs.

In reality, throwing more compute and data at models is not making them that much smarter. In fact, we are probably going to run out of good-quality public data because of the fall of Stack Overflow and smaller publishers that are going out of business.

In this article, let’s talk about a potential solution to many of these AI problems.

AGI is not coming

Even though I’m not a data science expert, I have worked with LLMs for a better part of 3-4 years now and have done tons of research into how they work and the field in general. This gives me some basis to comment on AGI and the future of AI, I think?

AGI is a pipe dream because LLMs are just prediction machines at the end of the day. Don’t get me wrong, the current iteration of this tech is really amazing and works really well in certain use cases, but it’s far from perfect and nowhere near what a human can do.

They have no worldview or real understanding of the data. They are trained on billions of examples, questions, answers, and fragments of information. They operate by breaking sentences into tokens, mathematically comparing inputs to learned patterns, and returning a human-like response through statistical grounding.

It’s a fancy algorithm; this is why AGI with the current architecture is impossible.

Secretly, AI companies have accepted this fact. They say many things in the media, but in reality, models are becoming commodities, and AI companies are just building ecosystems around these, so that they can lock in those subscriptions and keep you in their environment.

Think about it, why build Claude Code, a tool for developers, if AGI is going to replace all these developers, who then is going to pay the subscription fees?

AGI is not coming, and LLMs are becoming commodities; so, what’s next?

AGI is fueling public anxiety

People are scared of AI, let alone AGI, because every time you tune into the news, YouTube, TikTok, or some other media platform, you see big tech CEO’s making outlandish claims about how AI is so close to becoming AGI, and how they want everyone to use this tech, and if you don’t, you’re missing out and will likely be replaced by AI.

These claims alienate potential users rather than attracting them. Additionally, companies like Microsoft have been criticized for poor implementations of their AI offerings and basically just forcing AI into everything! even in contexts where it really adds no value, and makes the user experience suck so much more!

This pushes the average Joe towards two extremes: either using the technology inappropriately, such as relying on it for critical medical advice, or dismissing it altogether as yet another unreliable app that only works some of the time.

The truth is that AI is not as important as big tech CEOs would like you to think. Now, don’t get me wrong, it’s a powerful tool and can be really useful when used correctly, but it’s not as impactful as something like mobile phones or even the web.

Mobile phones are part of everyday life; we use them to text, communicate, socialize, consume media, and also as a business tool. The use cases are endless and widespread, and while I was born in a world where mobile phones didn’t originally exist, I still value this device enough to confidently say I just cannot function day-to-day without it!

I cannot say the same about AI. Sure, it makes my life a little easier, but its use cases are very niche, and for most people, they’re just using it as a Google++ or social media content generator, hardly life-changing!

I would consider it as a value-add rather than a full-blown platform on its own.

Tiny LLMs are the future

One of the biggest problems with LLMs is that they are GPU hungry. Your typical LLM, like Claude Opus, has been trained on trillions of tokens containing data from the entire public web and then some.

Yet I’m just using Claude code to generate some PHP or Python code; do I really need a model trained on trillions of tokens for that? The energy and infrastructure costs are significantly high, and it’s hard not to question whether this arms race toward AGI is worth it.

Instead, we should pursue a different approach: treat frontier models primarily as statistical engines, paired with a strong baseline of intelligence and tool-calling capabilities.

So basically, a kind of “Flash” or “Mini” model similar to GPT 5 MINI or Gemini 2.5 Flash. The goal would be a model capable of basic probabilistic generation and lightweight reasoning, without the cost and complexity of full frontier-scale model like Opus.

This will reduce the amount of GPU resources we need, making these models more scalable and affordable for mass adoption.

Knowledge packs can solve most issues

This brings me to the central question: could tiny LLMs replace larger models like Sonnet or Opus for most use cases? The challenge is that smaller models lack the extensive knowledge corpus of their larger counterparts, constraining their performance across many tasks. They're better suited for narrow, specialized use cases.

This is where knowledge packs come in. A knowledge pack is like a black box of pretrained weights; think of it like a plugin. You don’t need to know what makes up the plugin, or how it even works; all you need is a consistent API that the LLM can talk to, to retrieve the knowledge it needs as the need arises.

For example, a knowledge pack for the PHP programming language might be 20-30GB in size, which can comfortably fit on a modern laptop or computer. When you’re working in Laravel, you would then point the LLM to that knowledge pack, and it should naturally then scope its knowledge to that pack.

Whenever it needs to generate code or answer a question, it then only needs to look up information in that pack and not get confused by information that may look similar but is not related.

To give you a simple example: Let's take a random name: "Bob Barker." If I ask a large LLM, "Who is Bob Barker?", the model will generate a response based on patterns it learned during training from sources like LinkedIn, Wikipedia, Google searches, and countless other documents.

A frontier model might have 100B+ parameters because it was trained to handle everything from PHP programming to celebrity trivia to medical questions. Each time you prompt this model, it processes the input by running computations across all these parameters, requiring significant compute.

Now, imagine instead you have a much smaller base model, say 2B parameters, that was trained primarily on Wikipedia. This model would use a fraction of the computational resources at inference time simply because there are fewer parameters to process. It would also likely give more accurate answers because its training was concentrated on a specific domain, using a much narrower and more focused dataset (almost like fine-tuning).

This is where knowledge packs come in. Instead of one massive model trying to encode all human knowledge in its weights, you'd have a lean base model paired with specialized knowledge modules that can be loaded as needed. When you're asking about Bob Barker, you'd activate the "general knowledge" pack. When you're coding in Laravel, you'd load the "PHP/Laravel" pack and so on.

This isn't quite RAG - you're not doing runtime retrieval from a vector database, instead, you're swapping in pretrained weights or specialized model components that give the base model deep expertise in specific domains, while keeping the active computational footprint small.

The only problem is if you loaded a PHP knowledge pack, but the user asks a CSS question. What does the model do?

I think we can extend the tool-calling capabilities of these models to understand knowledge pack routing. Each knowledge pack would contain metadata that the model uses to determine how to route each prompt request. However, this mechanism would be built into the model itself and wouldn't pollute the context window (similar to how MCP can pollute the context window by publishing too many tools or resources).

Knowledge packs can fix funding roadblocks

Whether you talk about Google AI overview or ChatGPT, it’s a reality that most of this data was kind of “stolen” from publishers, even if they cite the authors, most end-users won’t even bother visiting the actual website. Everyone is obsessed with instant gratification.

LLM companies used public data that publishers, developers, artists, and authors, in general, have spent thousands of hours creating; models usually obscure the information enough so that it cannot be easily associated with the original work, and when it can be, they cite the content created as a source.

These small content creators never get a dime for their hard work, and many have to either downsized or shut down their website entirely because they just aren’t getting enough direct traffic anymore, and this is only going to get worse as the trust in LLMs grow.

Allowing independent publishers to publish their own knowledge packs provides a mechanism for them to get back some of this lost revenue.

I, as an individual, can choose to buy a knowledge pack from an open-source shop or subscribe to a monthly subscription to get the latest content for a small fee, instead of paying $20-$100 to these big tech companies that just never contribute back to these communities or individual artisans.

Sure, this probably will inflate the monthly $20 Claude subscription, but with the big players on board, I’m sure we can work out a competitive funding model.

Conclusion

Not to be repetitive, but AGI is not coming!

We need to move away from this big LLM idea and focus on building smaller, narrowly focused tiny LLMs that can be paired with specialized knowledge packs. This approach offers a path toward more sustainable, affordable, and ethically sound AI.

One that reduces computational waste, empowers content creators, and delivers better results for specific use cases. The future of AI isn't about building a giant AGI model that replaces humans, it's about using AI as a tool, optimized to help both free and commercial users drive real value from the product in their daily lives.

It’s about moving past the hype and capitalist ideals and building a future with AI responsibly!

I’ll keep this article updated as I go along, so please consider joining my newsletter. The purpose of the newsletter is just to auto email you whenever I add new content on the blog or update existing articles.

Model basics

In Django, we use a model class to represent the structure of a database table, and each field represents a column in that table. When you create an instance of this class, you effectively represent one row of data from that table.

Note: By default, Django creates the id field as the primary key, so you don't need to manually set a primary key field.

class ContactForm(models.Model): 
   full_name = models.CharField(max_length=100) 
   message = models.CharField(max_length=500) 
   contact_number = models.CharField(max_length=15, blank=True, null=True)
   email = models.CharField(max_length=255)
   date_submitted = models.DateTimeField(auto_now_add=True, null=True)

Blank versus null?

1. null - Can store NULL in the database. If False Then the DB column will be created with "not null", e.g. full_name VARCHAR(100) NOT NULL. Django default: False.

2. blank - Means the field can be empty in forms if set to True . Django default: False

When adding/changing model fields, you must run the following to generate a migration:

python manage.py makemigrations

If there are migrations, then also run:

python manage.py migrate

Run this to access your database's shell, so that you can use RAW SQL and navigate your database directly:

python manage.py dbshell

Django automatically assigns a primary key to your table, and the default field name is id. You can off course, change this by setting primary_key=True on another field in your model:

class Order(models.Model): 
    total_price = models.DecimalField(max_digits=18, decimal_places=2) 
    order_reference = models.CharField(max_length=50, primary_key=True)

If you want to represent a relationship between models, you can set this on children as follows:

class OrderItem(models.Model): 
    order = models.ForeignKey(Order, on_delete=models.CASCADE) 
    item_name = models.CharField(max_length=100) 
    item_price = models.DecimalField(max_digits=18, decimal_places=2) 
    quantity = models.IntegerField(default=1)

class UserProfile(models.Model): 
    user = models.OneToOneField(User, on_delete=models.CASCADE)

class Course(models.Model): 
    course_name = models.CharField(max_length=100)

class Student(models.Model): 
    student_name = models.CharField(max_length=100) 
    courses = models.ManyToManyField(Course)

models.CASCADE , models.DO_NOTHING (more options like PROTECT also supported) are constraits that tell Django what to do with children when you delete a parent. Cascade will delete all children, and "DO_NOTHING" will not delete anything.

Django will automatically slugify and create a table name using this format:

app_name_modelclassname

example: websites_contactform

You can change the table name if you need, as follows:

class ContactForm(models.Model):
    class Meta:
        db_table = "my_contact_form"

# table name in the db is: my_contact_form

User model - override the default:

class CustomUser(AbstractUser):
    username = None
    email = models.EmailField(unique=True, max_length=255)
    ... other fields

    USERNAME_FIELD = "email"
    REQUIRED_FIELDS = []

    def __str__(self):
        return self.email

# Settings file
  AUTH_USER_MODEL = 'accounts.CustomUser'

Model Queries

When you inherit from models.Model Django automatically sets up a default model manager named objects on every model class. Model managers are the interface between your Django code and the database; they allow you to query and manipulate data in a Pythonic way. Under the hood, the manager sits on top of a complex chain of classes that handles everything from building queries to communicating with the database backend.

⚠️ I simplified some of the SQL queries to make it easier to understand. Django optimizes and generates more explicit SQL than my simple "SELECT * " versions.

To perform queries, you can use methods of the manager as follows:

users = User.objects.filter(email="test@test.com")
orders = Order.objects.filter(status="paid", order_total__gte=500)

The filter method is basically translating your query to something similar in SQL:

SELECT * FROM users where email="test@test.com"
SELECT * FROM orders where status="paid" and order_total >= 500

# For simplicity, left out the full table name that Django would normally generate.

The __gte is a field lookup; these allow you to perform queries that are more advanced than a simple = . They support the typical math variants __gt , __lt , __gte , string matching: __icontains , in list lookups __in and many more operations.

In addition to filter You can also use exclude which also supports the same querying style but does the reverse, i.e. filter ensures results must match the criteria, and exclude ensures results don't match the relevant expressions passed to it.

By default filter/exclude will do an AND query, and you can chain multiple of these together because the manager class will create a QuerySet object (sort of like an in-memory wherestatement). The QuerySet is basically lazy-loaded, meaning Django doesn't actually fetch the data until you evaluate this by calling count , or perform some kind of evaluation like looping through the records.

filter(status="paid", order_total__gte=500)
 ---> status="paid" and order_total >= 500

You can make it a SQL OR by using the special Q object:

from django.db.models import Q
orders = Order.objects.filter(
    Q(status="paid") | Q(order_total__gte=500)
)

---> status="paid" OR order_total >= 500

Aggregate queries

While filter allows you to perform common where type queries in SQL, you sometimes need to do aggregates like SUM, COUNT, AVG etc... Count is fairly easy:

Order.objects.filter(
    Q(status="paid") | Q(order_total__gte=500)
).count() # just add .count at the end of any query

For other aggregate types, Django provides special classes for these, and a model manager method aggregate:

 from django.db.models import Avg, Max, Sum

Order.objects.aggregate(Max("order_total", default=0))
Order.objects.aggregate(Avg("order_total", default=0))
Order.objects.aggregate(Sum("order_total", default=0))

Aggregate will basically generate a similar query:

SELECT SUM(order_total) FROM orders;
# returns as a result: {"order_total__sum": 1500.00}

Essentially a single value, but what if you want to group by And count records? For example, you have a bunch of e-commerce merchants, and you want to know how many orders each merchant received. In this case, we can use annotate:

# logic
from django.db.models import Avg, Max, Sum, Count

merchants = Merchant.objects.annotate(Count("orders", distinct=True))
# Django uses the related_name field here for "orders"

for m in merchants:
    print(m.orders__count)

The model:

class Order(models.Model): 
    merchant = models.ForeignKey(Merchant, on_delete=models.DO_NOTHING, related_name="orders")

Example SQL:

SELECT merchant.*, COUNT(DISTINCT orders.id) 
FROM merchants
LEFT JOIN orders ON orders.merchant_id = merchant.id
GROUP BY merchant.id;

aggregate and annotate operate on a QuerySet So you can also chain filter and exclude before aggregating/annotating your data.

Order.objects.filter(status="paid").aggregate(Sum("order_total", default=0))

You can print the RAW SQL of any QuerySet by simply printing as follows:

query_set = User.objects.annotate(Count("order", distinct=True))
print(query_set.query)

# Output
SELECT "auth_user"."id", "auth_user"."password", "auth_user"."last_login", "auth_user"."is_superuser", "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", "auth_user"."date_joined", COUNT(DISTINCT "website_order"."order_reference") AS "order__count" FROM "auth_user" LEFT OUTER JOIN "website_order" ON ("auth_user"."id" = "website_order"."user_id") GROUP BY "auth_user"."id", "auth_user"."password", "auth_user"."last_login", "auth_user"."is_superuser", "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", "auth_user"."date_joined"

Generate fake test data

To generate fake data, you can use an excellent third-party library called Faker. If you do this in a loop, Faker will automatically generate dynamic values, which is perfect for testing, as you can generate hundreds, even thousands, of fake users and other objects.

from faker import Faker  # pip install Faker
from django.contrib.auth.models import User

fake = Faker()

user = User.objects.create_user(
    username= fake.user_name(),
    email=fake.email(),
    first_name=fake.first_name(),
    last_name=fake.last_name(),
    password=fake.password()
)

These are just basic fields; the library can generate loads more with all kinds of customizations.

Model managers

Whenever you perform a DB query, you will notice we always chain methods to the objects property on our model, e.g. : MyModel.objects.filter . The "objects" property is an instance of a manager class. As the name suggests, this object will "manage" all the interactions between high-level API's we developers use, such as filter, exclude (etc...) and low-level Django ORM objects that eventually build SQL queries and execute them on the database server.

For example, we can build a custom manager that would ensure every time we use filterWe automatically append a where clause such as: team_id=x.

Instead of using:

User.objects.filter(department="Accounting")

We could use:

User.for_team.filter(department="Accounting")

In the first query, the SQL will look similar to:

SELECT * from users where department = 'accounting'

However, the second query could look like:

SELECT * from users where department = 'accounting' and team_id = 2

A model manager example:

class TeamManager(models.Manager):
    def get_queryset(self):
        team_id = get_team_id()
        return super().get_queryset().filter(team_id=team_id)

class User(models.Model):
    for_team = TeamManager()

Remember queryset is a lazy representation of a database query, i.e., sort of like a virtual SQL statement that Django maintains in memory while you keep chaining filter, exclude and various other model methods.

So essentially, when you invoke the default model manager .objects It sets up a blank queryset , and manages all the changes to it until you finally execute that query on the database server.

So in this case, instead of returning a blank querysetWe give this queryset a where clause.

ℹ️ get_team_id() under the hood, is a custom function (i.e. not part of Django) that's using a ContextVar We set in our middleware, but this logic is beyond the scope of managers, so I'll not go further on that for now, but just assume that, regardless of whatever this function does, it returns a team_id, which we then use in our queryset.

Simple OTP generator

You can use this for 2-factor auth. Store the OTP in a temp table, and then send the user an email with the PIN for them to confirm. Encryption is for storage, so the PIN isn’t easily readable in the DB table.

import base64
import random
from hashlib import sha256

from cryptography.fernet import Fernet
from django.conf import settings


def generate_otp():
    return str(random.randint(100000, 999999))


def get_encryption_key():
    key_hashed = sha256(settings.SECRET_KEY.encode()).digest()
    return base64.urlsafe_b64encode(key_hashed)


def encrypt_otp(otp_code):
    key = get_encryption_key()
    fernet = Fernet(key)
    encrypted = fernet.encrypt(otp_code.encode())
    return encrypted.decode("utf-8")


def decrypt_otp(encrypted_otp):
    if not encrypted_otp:
        return None

    try:
        key = get_encryption_key()
        fernet = Fernet(key)
        decrypted = fernet.decrypt(encrypted_otp.encode())
        return decrypted.decode("utf-8")
    except Exception:
        return None

Decorator for protecting views

While middleware is usually better, sometimes you just need to run some logic only for a handful of views, in which case it might be easier to use a decorator.

def is_account_verified(view_func):
    @wraps(view_func)
    def _wrapped_view(request, *args, **kwargs):
        if not request.user.is_authenticated:
            return redirect("login")

        if request.user.user_profile.is_confirmed is False:
            request.user.user_profile.send_verify_email(request)
            return redirect("must-verify")
        response = view_func(request, *args, **kwargs)
        return response

    return _wrapped_view


####### Use in views.py #######
from accounts.decorators import is_account_verified

@is_account_verified
def change_password(request):
    pass

Auth views

Customizing the auth. If you want to use Django’s default auth system but not admin, you can extend and customize the existing auth to use your own template and your own custom register/login.

from django.contrib.auth import views as auth_views
urlpatterns = [
    path("login/", login_view, name="login"),
    path("register/", register_view, name="register"),
    path("logout/", logout_view, name="logout"),
    path(
        "password_reset/",
        auth_views.PasswordResetView.as_view(
            template_name="auth/password_reset_form.html",
            html_email_template_name="auth/emails/password_reset_email.html",
            subject_template_name="auth/emails/password_reset_subject.txt",
        ),
        name="password_reset",
    ),
    path(
        "password_reset/done/",
        auth_views.PasswordResetDoneView.as_view(
            template_name="auth/password_reset_done.html"
        ),
        name="password_reset_done",
    ),
    path(
        "reset/<uidb64>/<token>/",
        auth_views.PasswordResetConfirmView.as_view(
            template_name="auth/password_reset_confirm.html"
        ),
        name="password_reset_confirm",
    ),
    path(
        "reset/done/",
        auth_views.PasswordResetCompleteView.as_view(
            template_name="auth/password_reset_complete.html"
        ),
        name="password_reset_complete",
    )
]

def login_view(request):
    if request.user.is_authenticated:
        return redirect("dashboard")

    if request.method == "POST":
        username = request.POST.get("username")
        password = request.POST.get("password")
        user = authenticate(request, username=username, password=password)
        if user is not None and user.user_profile.is_active is False:
            messages.error(
                request, "Sorry, but your administrator has disabled your account."
            )
        elif user is not None:
            if user.user_profile.is_twofactor_enabled:
                request.session["2fa_user_id"] = user.id
                request.session["2fa_timestamp"] = timezone.now().isoformat()
                return redirect("send_2fa_email")
            else:
                login(request, user)
                if user.user_profile.needs_password_change:
                    return redirect("change_password")
                return redirect("dashboard")
        else:
            messages.error(request, "Invalid username or password.")

    return render(request, "auth/login.html")


def register_view(request):
    if request.user.is_authenticated:
        return redirect("dashboard")

    context = {}

    if request.method == "POST":
        email = request.POST.get("email")
        password1 = request.POST.get("password1")
        password2 = request.POST.get("password2")
        full_name = request.POST.get("name")

        context = {
            "email": email,
            "name": full_name
        }

        if password1 != password2:
            messages.error(request, "Passwords do not match.")
            return render(request, "auth/register.html", context)

        if len(password1) < 8:
            messages.error(request, "Password must be at least 8 characters long.")
            return render(request, "auth/register.html", context)

        if not re.search(r"\d", password1):
            messages.error(request, "Password must include at least one number.")
            return render(request, "auth/register.html", context)

        if not re.search(r"[A-Z]", password1):
            messages.error(
                request, "Password must include at least one uppercase letter."
            )
            return render(request, "auth/register.html", context)

        if not full_name:
            messages.error(request, "Oops! Please tell us your name?")
            return render(request, "auth/register.html", context)

        # probably want a more generic message
        if User.objects.filter(email=email).exists():
            messages.error(request, "Email already in use.")
            return render(request, "auth/register.html", context)

        try:
            user = User.objects.create_user(
                email=email, password=password1
            )
            full_name = full_name.split(" ")
            if len(full_name) > 1:
                user.first_name = full_name[0]
                user.last_name = " ".join(full_name[1:])
            else:
                user.first_name = full_name[0]
                user.last_name = ""

            user.save()
            messages.success(
                request, "Account created successfully. You can now log in."
            )

            return redirect("login")
        except Exception as e:
            messages.error(request, f"Error creating account: {e}")
            return render(request, "auth/register.html", context)

    return render(request, "auth/register.html", context)


def logout_view(request):
    logout(request)
    return redirect("login")

Management commands

# --- in app_name/management/commands
from django.core.management.base import BaseCommand


class Command(BaseCommand):
    help = "Some important task description here"

    def add_arguments(self, parser):
        parser.add_argument("--email", type=str, help="Email address of the user")

    def handle(self, *args, **options):
        email = options.get("email")
        if not email:
            email = input("Enter email address of the user: ")
        # .... rest of the code

        self.stdout.write(
            self.style.SUCCESS(f"Successfully did whatever I was suppose to do.")
        )

Dead simple background tasks without Celery complication

The purpose of this is to run simple tasks in the background; it’s not meant for high concurrency or complex jobs. For that, just use Celery. The beauty of this way is that there’s no external service to manage (besides the systemd script), so it’s easier to manage.

In an app “background_tasks”. models.py:

from django.db import models
from django.utils import timezone


class Statues(models.TextChoices):
    PENDING = "pending", "pending"
    IN_FLIGHT = "in_flight", "in_flight"
    FAILED = "failed", "failed"
    COMPLETE = "complete", "complete"


class Queues(models.TextChoices):
    HIGH = "high", "high"
    LOW = "low", "low"
    NORMAL = "normal", "normal"


class QueueTask(models.Model):
    job_data = models.JSONField("TASK", default=dict)
    status = models.CharField(max_length=20, choices=Statues, default=Statues.PENDING)
    scheduled_for = models.DateTimeField(default=timezone.now)
    queue = models.CharField(max_length=20, choices=Queues, default=Queues.NORMAL)
    callable = models.CharField(max_length=100, null=False, blank=False)

    @classmethod
    def queueTask(cls, job_data, callable, queue=None, scheduled_for=None):
        if callable is None:
            raise Exception(
                "Callable needs to be a task declared in background_jobs/tasks."
            )

        q = QueueTask()
        q.status = Statues.PENDING
        q.scheduled_for = timezone.now() if scheduled_for is None else scheduled_for
        queue = Queues.NORMAL if queue is None else queue
        q.callable = callable
        q.job_data = job_data
        q.save()

Example background_tasks/tasks/notification.py:

from django.conf import settings
from django.core.mail import EmailMessage
from django.utils import timezone

from background_jobs.models import QueueTask

def send_email(task: QueueTask):
    json_data = task.job_data
    json_data["to_emails"]
    for to_email in json_data["to_emails"]:
        email = EmailMessage(
            json_data["subject"],
            json_data["message"],
            (
                json_data["from"]
                if "from" in json_data.keys()
                else settings.DEFAULT_FROM_EMAIL
            ),
            [to_email],
        )
        email.content_subtype = json_data["format"] if "format" in json_data else "html"
        email.send()

In background_tasks/management/commands/worker.py:

import importlib
import logging
import time
from typing import List

from django.core.management.base import BaseCommand
from django.utils import timezone

from background_jobs.models import Queues, QueueTask, Statues

logger = logging.getLogger(__name__)


class Command(BaseCommand):
    help = "Process pending tasks from a specified queue"

    def handle(self, *args, **options):
        while True:
            try:
                tasks = QueueTask.objects.filter(
                    status=Statues.PENDING,
                    scheduled_for__lte=timezone.now(),
                    queue=Queues.NORMAL,
                ).order_by("-scheduled_for")[:20]

                for t in tasks:
                    self.execute_task(t)
            except Exception as ex:
                print(ex)
            finally:
                time.sleep(15 / 1000)

    def execute_task(self, task: QueueTask):
        """Execute a single task"""
        try:
            self.stdout.write(f"Executing task {task.id}")
            task.status = Statues.IN_FLIGHT
            task.save()

            module_name, function_name = task.callable.split(".")
            module = importlib.import_module("background_jobs.tasks." + module_name)

            getattr(module, function_name)(task)

            task.status = Statues.COMPLETE
            task.save()

            logger.info(self.style.SUCCESS(f"Task {task.id} completed successfully"))

        except Exception as e:
            task.status = Statues.FAILED
            task.save()
            logger.error(self.style.ERROR(f"Task {task.id} failed: {str(e)}"))

Then just set up a systemd on Ubuntu / Debian (/etc/systemd/system/worker.service):

[Unit]
Description=Worker daemon for my django app queue
After=network.target

[Service]
User=pythonapp
Group=pythonapp
WorkingDirectory=/home/pythonapp/myproject
ExecStart=/home/pythonapp/.venv/bin/python manage.py worker

Restart=on-failure
RestartSec=5s

Environment=DJANGO_SETTINGS_MODULE=myproject.settings

[Install]
WantedBy=multi-user.target

Then in Ubuntu:

systemctl daemon-reload 
systemctl enable worker.service
systemctl start worker.service

Check the status of the worker:

systemctl status worker.service

Now, throughout your application, you can then queue work like so:

QueueTask.queueTask(
    {
        "subject": subject,
        "message": message,
        "to_emails": [self.user.username],
        "format": "html",
    },
    "notifications.send_email",
)

NGINX to reverse proxy to Gunicorn worker:

server {
    listen 80;
    real_ip_header CF-Connecting-IP;
    server_name mydomain.com www.mydomain.com;

    # Redirect HTTP to HTTPS
    return 301 https://\(host\)request_uri;
}

server {
    listen 443 ssl;
    server_name mydomain.com
    real_ip_header CF-Connecting-IP;
    # SSL configuration
    ssl_certificate /etc/ssl/mydomain.com.cert;
    ssl_certificate_key /etc/ssl/mydomain.com.key;
    ssl_protocols TLSv1.2 TLSv1.3;

    # Security headers
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Static file serving
    location /static/ {
        autoindex on;
        alias /var/www/static/;
        expires 30d;
        access_log off;
    }

    # Proxy connections to Gunicorn
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 75s;
        proxy_read_timeout 300s;
        client_max_body_size 10m;
    }

    # Logging
    access_log /var/log/nginx/mydomain.com.access.log;
    error_log /var/log/nginx/mydomain.com.error.log;
}

Gunicorn worker config:

[Unit]
Description=Gunicorn daemon for myproject.com
After=network.target

[Service]
User=pythonapp
Group=pythonapp
WorkingDirectory=/home/pythonapp/myproject
ExecStart=/home/pythonapp/.venv/bin/gunicorn \
          --access-logfile - \
          --workers 8 \
          --threads 2 \
          --timeout 180 \
          --max-requests 1000 \
          --max-requests-jitter 100 \
          --bind 127.0.0.1:8000 \
          myproject.wsgi:application

Restart=on-failure
RestartSec=5s

# Environment variables if needed
Environment=DJANGO_SETTINGS_MODULE=myproject.settings

[Install]
WantedBy=multi-user.target

Forms - implement the change user password form

from django.contrib.auth.password_validation import validate_password
from django.core.exceptions import ValidationError as DjangoValidationError
from django import forms


class CustomPasswordChangeForm(forms.Form):
    current_password = forms.CharField(
        label="Current Password", 
        widget=forms.PasswordInput(attrs={'autocomplete': 'current-password'}),
        required=True
    )
    new_password1 = forms.CharField(
        label="New Password", 
        widget=forms.PasswordInput(attrs={'autocomplete': 'new-password'}),
        required=True
    )
    new_password2 = forms.CharField(
        label="Confirm New Password", 
        widget=forms.PasswordInput(attrs={'autocomplete': 'new-password'}),
        required=True
    )

    def __init__(self, user, *args, **kwargs):
        self.user = user
        super().__init__(*args, **kwargs)

    def clean_current_password(self):
        current_password = self.cleaned_data.get("current_password")
        if not self.user.check_password(current_password):
            raise forms.ValidationError(
                "Your current password was entered incorrectly."
            )
        return current_password

    def clean_new_password1(self):
        """Validate new password against Django's validators"""
        password = self.cleaned_data.get("new_password1")
        if password:
            try:
                validate_password(password, self.user)
            except DjangoValidationError as e:
                raise forms.ValidationError(e.messages)
        return password

    def clean_new_password2(self):
        password1 = self.cleaned_data.get("new_password1")
        password2 = self.cleaned_data.get("new_password2")

        if password1 and password2 and password1 != password2:
            raise forms.ValidationError("The two password fields didn't match.")

        return password2
    
    def clean(self):
        cleaned_data = super().clean()
        current = cleaned_data.get("current_password")
        new = cleaned_data.get("new_password1")
   
        if current == new:
            raise forms.ValidationError(
                "Your new password cannot be the same as your current password."
            )
        
        return cleaned_data

    def save(self, commit=True):
        password = self.cleaned_data["new_password1"]
        self.user.set_password(password)
        if commit:
            self.user.save()
        return self.user

@login_required
@require_http_methods(["POST"])
def change_password(request):
    if request.headers.get("X-Requested-With") == "XMLHttpRequest":
        form = CustomPasswordChangeForm(request.user, request.POST)
        if form.is_valid():
            form.save()
            update_session_auth_hash(request, form.user)
            return JsonResponse(
                {"success": True, "message": "Your password was successfully changed."}
            )
        else:
            return JsonResponse({"success": False, "errors": form.errors}, status=400)

    return JsonResponse({"success": False, "message": "Invalid request"}, status=400)

Setting up S3 storage in settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django_redis',
    'shared',
    "storages",   <----
]

pip install django-storages

# Public S3 Storage Configuration
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME', 'mybucket')
AWS_S3_REGION_NAME = os.getenv('AWS_S3_REGION_NAME', 'fsn1')
AWS_S3_ENDPOINT_URL = os.getenv('AWS_S3_ENDPOINT_URL', f'https://{AWS_S3_REGION_NAME}.your-objectstorage.com')
AWS_S3_CUSTOM_DOMAIN = os.getenv('AWS_S3_CUSTOM_DOMAIN', f'{AWS_STORAGE_BUCKET_NAME}.{AWS_S3_REGION_NAME}.your-objectstorage.com')
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}
AWS_DEFAULT_ACL = 'public-read'
AWS_QUERYSTRING_AUTH = False
AWS_S3_FILE_OVERWRITE = False

# Private S3 Storage Configuration
PRIVATE_BUCKET_NAME = os.getenv('PRIVATE_BUCKET_NAME', 'myprivatebucket')
PRIVATE_S3_REGION_NAME = os.getenv('PRIVATE_S3_REGION_NAME', 'nbg1')
PRIVATE_S3_ENDPOINT_URL = os.getenv('PRIVATE_S3_ENDPOINT_URL', f'https://{PRIVATE_S3_REGION_NAME}.your-objectstorage.com')
PRIVATE_S3_CUSTOM_DOMAIN = os.getenv('PRIVATE_S3_CUSTOM_DOMAIN', f'{PRIVATE_BUCKET_NAME}.{PRIVATE_S3_REGION_NAME}.your-objectstorage.com')

STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3boto3.S3Boto3Storage",
        "OPTIONS": {
            "access_key": AWS_ACCESS_KEY_ID,
            "secret_key": AWS_SECRET_ACCESS_KEY,
            "bucket_name": AWS_STORAGE_BUCKET_NAME,
            "region_name": AWS_S3_REGION_NAME,
            "endpoint_url": AWS_S3_ENDPOINT_URL,
            "custom_domain": AWS_S3_CUSTOM_DOMAIN,
            "object_parameters": AWS_S3_OBJECT_PARAMETERS,
            "default_acl": AWS_DEFAULT_ACL,
            "querystring_auth": AWS_QUERYSTRING_AUTH,
            "file_overwrite": AWS_S3_FILE_OVERWRITE,
            "location": "",
        },
    },
    "private": {
        "BACKEND": "storages.backends.s3boto3.S3Boto3Storage",
        "OPTIONS": {
            "access_key": AWS_ACCESS_KEY_ID,
            "secret_key": AWS_SECRET_ACCESS_KEY,
            "bucket_name": PRIVATE_BUCKET_NAME,
            "region_name": PRIVATE_S3_REGION_NAME,
            "endpoint_url": PRIVATE_S3_ENDPOINT_URL,
            "custom_domain": PRIVATE_S3_CUSTOM_DOMAIN,
            "object_parameters": AWS_S3_OBJECT_PARAMETERS,
            "default_acl": 'private',
            "querystring_auth": True,
            "file_overwrite": AWS_S3_FILE_OVERWRITE,
            "location": "",
        },
    },
    "staticfiles": {
        "BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
    },
}

MEDIA_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/'

Global shared template tag

Sometimes you want to access branding information, and other important information in templates, so instead of hardcoding them or repeating yourself all the time. Put those settings in the settings.py and then reference them using this shared template tag.

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        'APP_DIRS': True,
        'OPTIONS': {
            "builtins": [
                "shared.templatetags.global_settings_tags",  <------
            ],
            'context_processors': [
                'shared.context_processors.global_settings',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

# Only allow these settings to be accessed from template tag
ALLOWED_TEMPLATE_SETTING_ACCESS = {
    'APPLICATION_LOGO',
    'APPLICATION_NAME',
    'CONTACT_EMAIL',
    'SUPPORT_URL',
    'MEDIA_URL',
    'STATIC_URL',
    # Add other settings you want templates to have access to
}

In shared/templatetags/global_settings_tags.py

from django import template
from django.conf import settings

register = template.Library()

@register.simple_tag
def get_setting(name):
    name = name.strip()
    if name not in settings.ALLOWED_TEMPLATE_SETTING_ACCESS:
        return ""

    return getattr(settings, name, "")

Now, anywhere in your templates, you can access:

   <img src="{% get_setting 'APPLICATION_LOGO' %}" alt="logo" />

Note: You probably want to break up your settings file into multiple config files. For larger projects, I normally create a “/config” folder and split up my settings files into there.

Django and Tailwind

tailwind.config.js at the root of your project

/** @type {import('tailwindcss').Config} */
module.exports = {
  content: [
    "./templates/**/*.{html,js}"
  ],
  theme: {
    extend: {
      colors: {
        primary: '#5DDED6',
        secondary: '#8b5cf6',
        dark: '#1e293b',
      },
      fontFamily: {
        sans: ['Inter', 'sans-serif'],
      },
      animation: {
        'float': 'float 3s ease-in-out infinite',
      },
      keyframes: {
        float: {
          '0%, 100%': { transform: 'translateY(0)' },
          '50%': { transform: 'translateY(-10px)' },
        }
      }
    },
  },
  plugins: [],
}

postcss.config.js

module.exports = {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
}

package.json:

{
  ----
  "scripts": {
    "build:css:prod": "tailwindcss -i ./static/css/app.css -o ./static/build/app.css --minify",
    "watch:css": "tailwindcss -i ./static/css/app.css -o ./static/build/app.css --watch",
    "watch:django": "python manage.py runserver",
    "dev": "npm-run-all --parallel watch:*"
  },
  "devDependencies": {
    "@fortawesome/fontawesome-free": "^6.7.2",
    "autoprefixer": "^10.4.21",
    "postcss": "^8.5.3",
    "tailwindcss": "^3.3.3"
  },
  "dependencies": {
    "npm-run-all": "^4.1.5"
  }
  ----
}

npm run dev  ~ to hot reload python and tailwind
npm run build:css:prod  ~ for production builds

Meh 😐, it’s okay, but it's better than GPT-5 for sure, and it’s incremental just like most other models we’ve seen this year.

When the dust settles and we get past all the hype, what really matters is how reliable and cost-effective these models are for day-to-day use, not just coding but also business use cases.

In this article, I want to talk about some lesser-known models that all the YouTubers and the media in general miss.

Gemini flash

My favourite model is Gemini 2.0 Flash for general machine learning and not-so-complicated tasks like RAG chatbots.

Flash is super fast compared to Anthropic’s Haiku and OpenAI mini models, yet feels much smarter than nearly all other mini models I have worked with.

I’m using Flash models for:

• Voice AI tasks.

• Rag chatbots.

• Categorization of products.

• Rewriting search queries.

• Various text generations.

Back in the day, GPT4o-mini was the goat of small models, but alas, in the last year or so, Google has just woken up and dominated the small model space.

PHI-3 / PHI-4 models

If you have a small 8GB - 16GB consumer GPU on hand and want to run a local model, then these PHI models are impressive. I use them for simple classification and text generation tasks. They generally perform well.

Sure, they are not as smart as the Gemini models, but if you use this model in combination with a semantic search or some other process, for example, categorization, you could do a semantic search to find the most relevant list of categories, then ask the model to pick the most appropriate.

This will save you some per-token costs and give you a decent level of accuracy.

Z.Ai GLM 4.6

It’s become a common practice for developers to use AI coding assistants like Claude Sonnet, Cursor, Windsurf, and so forth.

I myself love Sonnet and use it often, but Sonnet can be expensive and slow at times, which really impacts your flow state, not to mention all the daily and weekly limits if you’re using one of the subscription plans.

I still write and plan the “complex” stuff myself, but there’s just some boilerplate 🥱 brain-dead CRUD stuff that I just don’t need to write myself anymore; in the past, I would use Sonnet for these tasks, but of late have been really impressed by how much mileage I got out of GLM 4.6.

Z.ai subscriptions start from $6 a month, or you can use something like OpenRouter instead, and it’s fairly capable. I’ve generated some good Python, PHP, and even Golang code with the 4.6 model (using cline in VSCode).

You still need to review and clean up the code to an extent, like with Sonnet, but the 4.6 model seems fairly good with clearly defined tasks, tailwind stuff, and just CRUD-type low-thinking tasks.

Sure, Sonnet is still better at the more complicated stuff, but anyway, complex problems are fun for me, so I just use my brain for those 🙃.

Minimax models

Like GLM 4.6, the Minimax models are also very capable of agentic and coding tasks, and comparatively, the price is way cheaper than Sonnet. On OpenRouter, it’s like $2 per million output tokens.

Similar to GLM 4.6, it’s not as smart as Sonnet; however, remember that big ol brain you have! It’s usually good enough for most code generation tasks and can also be used as a smarter alternative to GPT-5 or any of the GPT models.

Let’s be honest, OpenAI is just not as good as it once was. Their API response times are not great, and most of these other alternative models have caught up.

OpenRouter

Much of what I am referencing here are models that I use through OpenRouter, which also happens to be one of my favourite AI platforms ever!

If you don’t know, OpenRouter is basically a “Router for AI”. Instead of you having to create accounts with OpenAI, Anthropic, Google, etc, you can simply register with them and buy credits. Those credits can then be used across hundreds (if not thousands) of models.

So, for example, in one project using just a single SDK implementation, you can swap between Gemini, Anthropic, Z.ai, OpenAI, Mistral, and many more providers by just changing the model name!

They also allow for multiple providers. Take, for example, GLM 4.6 has about 9-10 different companies that host the model, so essentially, if one provider is experiencing an outage, OpenRouter will automatically route your API call to one of the other providers. You can also optimize for costs to pick the cheapest provider!

Okay, this is sounding like an OpenRouter ad now, totally not (wouldn’t mind if they sponsored me though 😉), it’s really cool!

The only problem is that sometimes the API is slower compared to using the mainstream provider directly, and there is a small markup they add to your token usage to obviously make a profit, but generally, by using different providers and models, you end up saving money.

This will be a quick guide, purely meant as self-documentation to help me remember. I found it a bit annoying to find good documentation, it seems like the wild, wild west 🙃! many different documentation sources, and they all seem to do things differently.

PS: if you’re a Next.js pro, feel free to suggest a better way of doing things. I’m constantly looking to follow best practices wherever I can.

What are Server Components?

I personally think this is a game-changer, purely because I built many SPAs and understand the pain of maintaining an API, dealing with CORS, and context switching.

In Next.js, you can run React code on the server side. This means that Next.js will pre-compile the React code and then send the result back to the browser. So, for example, in the past, to fetch data in a component, you would need a separate API and boilerplate code like this:

"use client"
import { useEffect, useState } from "react";

interface Post {
  id: number;
  userId: number;
  title: string;
  body: string;
}

export default function Home() {
  const [posts, setPosts] = useState<Post[]>([]);

  useEffect(() => {
    const loadPosts = async () => {
      try {
        const response = await fetch('https://jsonplaceholder.typicode.com/posts');
        if (!response.ok) {
          throw new Error(`HTTP error! status: ${response.status}`);
        }
        const apiPosts: Post[] = await response.json();
        setPosts(apiPosts);
      } catch (error) {
        console.error('Failed to fetch posts:', error);
      }
    };

    loadPosts();
  }, []);

  return (
    <div>
      {posts.map(post => (
        <div key={post.id}>
          <h3>{post.title}</h3>
          <p>{post.body}</p>
        </div>
      ))}
    </div>
  );
}

It's such a pain. You have to use state and useEffect. Plus, you actually need an API. Here, I’m just using a public API, but for a real-world application, you’d need to build that API for your internal business data using Express.js or some other backend framework (You could also use Next.js API routes).

With Server Components We can greatly simplify this code; we don’t need to build API endpoints or set up new routes; we can just create a simple function and call it in our component:

interface Post {
  id: number;
  userId: number;
  title: string;
  body: string;
}

async function loadPosts() :Promise<Post[]> {
  const response = await fetch('https://jsonplaceholder.typicode.com/posts');
  return response.json();
}

export default async function Home() {
  const posts = await loadPosts();

  return (
    <div>
      {posts.map(post => (
        <div key={post.id}>
          <h3>{post.title}</h3>
          <p>{post.body}</p>
        </div>
      ))}
    </div>
  );
}

You probably want to separate the API call into its own actions/xyz.ts file to keep things clean and this is an oversimplified example to demonstrate the concept, so it’s just got the bare minimum.

As you can see, this is a lot simpler, and since the template is pre-compiled with the data from our fetch call, by the time it reaches the browser, the data is already present, so there’s no need to use useEffect or state.

By default, all components are ServerComponents, If you need to use React hooks like useEffect, useState Or any JavaScript events, then you can force the component to behave just like a regular React component by adding ”use client” .

Zod - what?!

This is not a Pokémon or Transformer 😂 or (Zordon from Power Rangers). TypeScript is a superset of JavaScript; thus, types don’t work at runtime, since the browser cannot understand TypeScript. So when it comes to forms, you can’t actually use an interface to ensure the types are valid.

You still need validation at runtime to provide meaningful error messages and also ensure the data is in the correct format before saving to a database, hence the mighty zod :

import {z} from 'zod';

export const UserSchema = z.object({
    name: z.string().min(3, {"error": "Please fill in a valid name."}),
    email: z.email({ pattern: z.regexes.html5Email })
});

export type User = z.infer<typeof UserSchema>;

So basically, UserSchema It’s just an object with some validation rules. Now, since Zod returns a validation object, it can’t be used as a type throughout your code. For example, in my server action, I need to type the input form data so it’s usable in my code.

'use server'
import { User, UserSchema } from "@/lib/types"

export const saveUser = async (formData: User) => {
   // other fun stuff here
}

To get an interface type out of Zod, simply do the following:

export type User = z.infer<typeof UserSchema>;
// then you can use fields like user.name, user.email etc...

Finally, you can use Zod to validate user input as follows in your server action.

export const saveUser = async (user: User) => {
    const valid = UserSchema.safeParse(user);
    if (!valid.success) {
        console.log(valid.error);
        return;
    }
}

Dealing with forms using React Hook Form

Forms are one of the most common things in any app, but they’re also one of the most annoying things to deal with because you need to validate the form data, show meaningful error messages, preserve the form state, etc…

Luckily, there’s a powerful form library that just makes working with forms that much easier; it’s not part of Next.js by default, so you need to install it as follows:

pnpm install react-hook-form

Now, we can make a form to save a user (a very simplistic example). We use a client-side component in this instance because we want to show errors and prevent the form from clearing its fields. We do validation on the client first before even sending the form to the server, so it’s a much better user experience.

'use client';

import { saveUser } from "@/actions/forms";
import { UserSchema, User } from "@/lib/types";
import {zodResolver} from '@hookform/resolvers/zod';
import {useForm} from 'react-hook-form';

export default function Home() {
  const form = useForm<User>({
    resolver: zodResolver(UserSchema)
  });

  const onSubmit = async (data: User) => {
    const result = await saveUser(data);
    alert(result);
    // result can anything you want to return from the server side.
  };

  return (
    <div>
      <form onSubmit={form.handleSubmit(onSubmit)}>
        <input {...form.register('name')}
        type="text"
        name="name" 
        className="bg-white border-2 mb-2 mt-2 text-slate-800"
         /> <br />

        <input 
        {...form.register('email')}
        type="email" 
        name="email"  
        className="bg-white border-2 mb-2 mt-2 text-slate-800"
        />

      <br />
          {form.formState.errors.email && (
          <p className="text-red-500">{form.formState.errors.email.message}</p>
        )}
        <br />

      <input type="submit" 
        value="SEND" 

        className="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded"
        />
      </form>
    </div>
  );
}

Breaking it down:

  const form = useForm<User>({
    resolver: zodResolver(UserSchema)
  });

We create a strongly typed form and link it to Zod. This ensures each field is checked against the schema. You can bind each of the form elements using this function …form.register(‘name’) so that when Zod runs, it can access the elements’ values and other properties.

       <input {...form.register('name')}
        type="text"
        name="name" 
        className="bg-white border-2 mb-2 mt-2 text-slate-800"
         />

Here’s where the magic happens:

 onSubmit={form.handleSubmit(onSubmit)}

In this call, before the data is sent to the Server Action function to handle saving the user, our form object will validate the Zod schema first. If there are any errors, it will not trigger the onSubmit function and instead just fill the errors in our form state, which we can then access as follows:

      <br />
          {form.formState.errors.email && (
          <p className="text-red-500">{form.formState.errors.email.message}</p>
        )}
        <br />

saveUser ~ is what we call a Server Action i.e., a function that components (both client and server) can call to perform some task on the server side.

In my project, I created a folder called actions which basically contains all my server actions.

Very Important: We need to ensure that the use server directive is always present in our server actions file, usually at the top of the file. This prevents the code from being bundled in the JavaScript code sent to the browser, which ensures a smaller JS file for the client to download. Furthermore, it prevents leaking any sensitive information to the browser, such as API keys and any other information you want to keep private.

"use server"

import { User, UserSchema } from "@/lib/types"
export const saveUser = async (user: User) :Promise<string> => {
    const valid = UserSchema.safeParse(user);
    if (!valid.success) {
        return valid.error.flatten.toString();
    }

    // do some db stuff
    return "Success";
}

If you came from the jQuery era like me, you’ll realize how much cleaner this is. We don’t need a 101 if statements to validate each form element, we can just use proper typing and let Zod do the rest.

You are worried about AI taking your job or what an AI-powered future may look like.

If you fit into one of the above categories, then this article is for you.

PS: The Hitchhiker's Guide to the Galaxy - Wikipedia ~ Inspired by this insanely cool movie.

What is AGI?

Let’s get this out of the way first. Nobody seems to have a clear idea of what AGI actually means, but here’s a general guideline.

AGI, at its core, is a goal post shift; previously, “AI” was thought of as being the all-intelligent machine that could think on the same level or even at a higher level than human beings.

AI companies started calling their “chatbots” (I’m using chatbots loosely for now) AI, thus AI lost the meaning it once had, and hence why they coined this new term “AGI”.

AGI, therefore, simply means the following:

• A machine that can understand information within the context of the real world.

• A machine that can learn and adapt to perform just about any task a human can do.

Wait! Can’t ChatGPT already do this?

No, ChatGPT can retrieve information, it can generate some pretty cool answers, and answer you in a way that you could mistake it for a human, but in reality, this is just a clever algorithm. ChatGPT itself doesn’t understand the meaning of the text it generates.

Here’s an example:

What I asked ChatGPT: “Draw me a clock with the hands showing the time as 12 minutes past 5 pm.”

What it generated:

The algorithm picked up that I want an image of a clock, but it got the time horribly wrong. Instead of responding with: “I am sorry, but I cannot draw those numbers on the clock,” or something like that, it still went ahead and drew the clock.

This is because it doesn’t fully understand the instruction; it’s looking for similar information in its vast knowledge base and just retrieving something that’s most similar. There is some fancy science going on to understand meaning to an extent, but still, it does not truly understand anything besides pattern matching.

Even after I corrected ChatGPT: “Uhm, but the time was wrong, please show the time 5:12 pm”, I still get:

AGI would therefore be aware of the question hollistically, and be able to draw the correct image, and if it can’t, it would recognise that it doesn’t have that ability, and try to learn that skill or just respond with: “I don’t know how to do that yet…”.

What is an LLM?

LLM stands for “Large Language Model”. This is basically what most refer to as “AI” or “AI Chatbots”. These are powerful programs that are trained on large pieces of information from around the web; they have a “brain” filled with billions of documents and pieces of information.

When you ask a Chatbot a question, it’s basically doing something similar to “Google Search”. The Chatbot breaks down your question into something called “tokens”, which are mathematical representations of words.

These tokens now give the Chatbot a superpower that goes beyond a simple keyword search, in that it can use math to figure out the relationship of words based on how these tokens are arranged, how far apart they are from each other, and so on.

So essentially, it takes your question and does a similarity search to find the most relevant tokens in its knowledge base, then uses those tokens together with some clever algorithmic magic to figure out the most likely answer that would resolve your query.

Now, complex LLMs like ChatGPT do a lot more magic in the background than just mathematical formulas, but at the end of the day, it’s still just a program that applies an algorithm.

What is a model?

Now that we have an understanding of how LLMs work, this is basically a continuation of the last question: “What is an LLM?”

AI companies like OpenAI spend months collecting data from various sources, and they clean that data by using various data science techniques to ensure it’s as accurate as possible.

The goal of this process is to build a knowledge base on which they can train the AI. You will notice every 6 months or so, there’s a new version of ChatGPT: “o1”, “4o”, “GPT-3”, “GPT3.5”, “GPT-5”, etc…

Training or teaching AI is an expensive venture; it takes time because we are dealing with billions of data points. Thus, a model is a program, a “black box” filled with layers and layers of information (known as neural networks).

The model provider will feed this program with the information over a period of time. The program will then store that information in its neural networks (a bit like memorizing information for an exam) and apply various mathematical calculations to make sense of the data. This process can take months; once it’s complete, the program is then frozen in time, its knowledge and understanding frozen on that dataset, thus the “GPT-5”, “o1”, etc, is just a label of the box based on which dataset it was trained on.

Furthermore, to clarify, if you trained the program when Obama was the president of the US, then asked the model, “Who is the current president? “ In 2022, it’s going to say “Oboma” or “My knowledge cut off is up until 2016, therefore I cannot answer that question”.

At every new version, the AI companies may discover new techniques and improve upon the dataset, which ultimately leads to a smarter model. This is why they name them at the end of each training cycle.

It may be possible that “GPT-5” is the smartest model at this point, but for a particular task, “GPT4o” might still perform better; this is why they don’t just overwrite the old model. It actually happened recently, where “GPT-5” answered questions more robotically; even though it was smarter, it seemed less human, thus people still preferred “GPT4o”.

While the model itself may not have memory beyond its cutoff point; most modern Chatbot systems like ChatGPT or Claude do have access to the internet and other data sources like RAG, which allows them to pull up to data information but if you discounted the model from these sources, it’s going to respond with the dataset version of it’s information.

What is hallucination?

I am sure you have experienced this before; if not, you will at some point when interacting with a chatbot. You ask the bot a question, it gives you a long, detailed answer full of confidence, but when you fact-check the answer, it’s either completely or partially incorrect.

Helliplication is basically when the algorithm predicts the answer incorrectly, which is bound to happen with current models, regardless of how smart they are.

What is RAG?

I covered this extensively throughout my blog with coding examples as well, so I won’t dive too deeply into this subject matter, but will just give you a general idea of what this means. If you are interested in a more technical understanding, please check out one of my earlier articles: How to build a PDF chatbot with Langchain 🦜🔗 and FAISS

RAG stands for “Retrieval Augmented Generation,” which is a fancy way of saying that you can supplement the model’s memory with your own dataset.

For example, if I had to ask ChatGPT, “What is the price of the latest iPhone?” It gives me pricing in dollars and information from foreign sources. I am South African, and I’m more interested in the “Rand” price and local stores.

To fix this, I can upload a PDF via the ChatGPT interface detailing the information from local retailers, and then re-ask it the same question, and tell it to find the answer from that PDF. This time round, it’s going to give me the correct answer based on the information in that PDF.

This is incredibly useful to websites that use Chatbots, because if I have an e-commerce store, I can build a RAG system that feeds product data from my store to the Chatbot; thus, even though I’m still using ChatGPT via the API, I can scope its response to only come from my RAG information and suggest only products from my store.

Can you imagine being on takealot.com, and then getting product recommendations from amazon.co.za? This will be a disaster, giving a competitor free advertising!

Therefore, RAG allows us some control of the Chatbot’s knowledge retrieval.

Inference and other common terminology

Some common terminology you’ll often hear:

• Tokens: When models ingest your prompt, they don’t read it as a sentence or word; instead, every word gets translated into a mathematical representation, and these representations are known as tokens. 1 token roughly equates to 4 characters, but this may vary depending on the word or phrase.

• Epoch: An epoch is one complete pass through your entire dataset. You normally would train the model via more than one epoch, because the second, third, fourth times, etc. It sees the data, it can apply optimizations, and improve its learning. Of course, too many Epochs can hurt accuracy, so this requires some balancing efforts.

• Inference: is when you actually ask the model a question, so this is a trained model, and each time you “prompt” the model, it needs to convert your request into tokens and do a whole bunch of math to determine the final response back. This process does require GPU resources as well, but it is not as computationally expensive as training.

• Model weights: After training a model, it generates something called “weights,” which are basically mathematical patterns derived from your training data. Since models are essentially predicting the next best word, phrase, or sentence, they use these weights to influence which sentences or words to return based on your input prompt.

• Neural network: Inspired by the human brain, which is made up of millions of neurons. Let's call them "micro-computers" that arrange together to form a chain, a network, and together these "micro-computers" can digest, analyze, and parse information in an intelligent way, leading to some outcome like making you smile, or crying, or running, etc. In the same way, models have neural networks made up of many layers, and each layer processes and transforms the information before passing it to the next layer. Collectively, they work together to build complex patterns and ultimately generate tokens that we get back as text or images, or even video.

• Model parameters: You’ll often see “7B”, “12B”, “24B”, 70B etc, next to model names. These refer to the number of parameters a model contains (in one billion units, so 7 billion, 12 billion, etc). Parameters are simply the number of weights and neural connections the model contains in its neural networks. Usually, the more parameters = smarter model, but this is not always the case because various factors can influence intelligence, including training process, quality of the dataset, and so forth.

What fine-tuning?

Like OpenAI or Anthropic, you too can train your own model on your own dataset. Usually, it would be crazy expensive to start from scratch, so it’s much easier and more cost-effective to just take an existing model and train it further using your own data.

This process will add or replace knowledge in the existing model to make it behave in a certain way. Currently, if you talk to Claude or ChatGPT, it’s conversational, but you may want a model that just follows a particular sequence.

This can be useful for restaurant ordering or booking car repair appointments, or just about any kind of niche task. You can give the model a few hundred / thousand examples, teaching how you want it to respond, and then the model will follow your examples rather than generic answers.

The benefits of fine-tuning:

• Faster inference. With RAG, you need to query a vectorstore, then pass that information to the model. This could add some speed overhead to responses. With Fine-tuned models, you don’t necessarily need to fetch extra data since everything is already baked into the model (you can still use RAG with fine-tuned models if needed).

• Cheaper inference. Not always the case, but for niche tasks, you could use much smaller models like “tinyllama” which can run on smaller GPUs.

• Better performance. While RAG works well, it doesn’t always lead to accurate results. With fine-tuning, you could get much more accurate responses depending on your data and use case.

Will AI take my job?

While AI can automate some tasks and help humans speed up their workflow, AI is not a drop-in replacement for human beings, purely because of the distinction made in my first point of “AGI”.

I am not saying there won’t be job losses; of course, with every new platform shift, there will always be a percentage of jobs no longer needed. In most of these cases, it’ll just require upskilling and pivoting of the individuals, rather than mass unemployment.

For example, in the past, you would need a frontend developer to design landing pages. With tools like Claude code, you could potentially spin up a world-class landing page in minutes without a frontend developer.

Does this mean the end of all frontend developers? No! Not all websites are simple landing pages; some need multi-stage forms, integration with legacy systems, and complex workflows. AI is simply not smart enough to handle all of these.

Thus, these developers would probably have more time to actually sit with clients or expand their skills into other domains like backend development, or even end up making way more money! Because they can automate more of their job, thus allowing them to push out more projects in a shorter space of time.

Not fun for my parents, as you can imagine 🙈

Nonetheless, this inquisitiveness has helped me a ton throughout my career. I love building stuff, but I love tearing things apart even more! I am never content with “It just works.” I always ask why.

As a software engineer, this is a vital skill to have, especially now in the modern era of AI, where code is just being generated willy-nilly. Understanding computer fundamentals will help you immensely in the long run to sustain a long and fruitful career in tech.

The goal of this article is to make you stop and think for a second about your code and what is actually happening when you hit that “build,” “compile,” or “run” button.

ℹ️ I will use a mixture of languages to demostrate some concepts (Usually Python and Golang), you do not need to understand these languages just yet, since the goal is to teach you how to think like an Engineer, such that you can apply this knowledge to just about any language.

Programming is just 1s and 0s, essentially electrical signals

Computer hardware can only understand one language: either turn on the flow of electricity in a pathway or turn it off.

Thus, regardless of the programming language you use, everything gets compiled down to machine code. Which is a set of instructions your operating system eventually passes down to the hardware in your computer as electrical signals.

Now, obviously, you cannot send raw 1s and 0s to the hardware; this is why we have operating systems and something known as machine code.

Operating systems are responsible for actually taking the machine code and converting it into electrical signals. Not all devices have an operating system; some just have smaller programs, often referred to as embedded systems. Nonetheless, they perform a similar role to that of a big OS like Windows, Linux, or Mac OS.

Machine code is the lowest level of software abstraction, and the last mile before code becomes electricity inside transistors. Essentially, machine code is a chain of ones and zeros, arranged together in a particular sequence.

Think of a computer monitor. It’s made up of millions of pixels of colors, i.e., dots. When we need to print the words “Hello World” to the screen, we send the OS machine code that may look like this: “1111 1101 1011 1111 1101 1111 1011” (Of course, I just made up this sequence for demonstration purposes).

Using this sequence, the system determines which pixels need to light up to form the shapes of "H-e-l-l-o W-o-r-l-d" according to the selected font, size, and color.

When you write a program, your code looks similar to written languages like English, albeit with a bit of math and symbols, but essentially it’s human-readable. This is by design, because we humans cannot manage thousands of 1s and 0s; it’ll be almost impossible to read and write accurately.

This is why we invented programming languages; they offer a high level of abstraction, allowing programmers to write code in a readable language, but also write code in a structured way so that it can be later converted into electrical signals.

Translator programs

Since programming languages aren’t written in machine code, the Operating System itself cannot understand your code. Instead, we need a translator program, something that can read your code and convert it to machine code.

Whenever you start working with a new programming language, you’ll often need to download and install some kind of runtime program like the Python interpreter or Node, or the JVM, and so forth.

While most of these runtimes come with various libraries and tools, at the heart of each runtime is a translator program of some sort.

There are many variations of translator programs, but generally, these are the most common:

• Compilers. In Golang, for example, the compiler runs once and generates what is known as an “executable”. This is your entire program in a binary format. To distribute your program, you no longer need the source code (the code you type in your coding editor) or the runtime to run the executable program, since it’s now machine code and can be executed directly by the Operating System.

• Interpreters. In the case of languages like Python, you do not build a single executable. Instead, you need to distribute your source code files, and in order for the target machine to run them, it needs to have the Python interpreter installed. While the interpreter may convert the program to an intermediary format like “bytecode” (we’ll talk about this soon), it’s not machine code and still needs processing to become machine code. So basically, interpreters read and convert your program’s code to machine code at runtime, hence why Python is much slower than Golang in terms of execution speed.

• JIT compilers. A best of both worlds, Node.js is a good example. When you distribute your program, it’s distributed as source code. When the program is first run, the JIT compiler will analyze your code and convert frequently used portions into binary code similar to a compiler. It further monitors and optimizes the code as the program is running.

So, in summary, translator programs like the Golang compiler are responsible for taking code you write in your code editor and converting that to machine code, such that your operating system can then execute and eventually convert those to electrical signals.

💡 “Parsing” is a step in the tranlation process of your code into an executable program. The translator program reads your code and analyzers it against the syntax rules of the programming language being used. During this process the parser will also throw errors if any exist in your code. Which can crash your program or cause the compiler to not generate an executable binary, depending on the type of translator program.

So, what is “Byte Code”

In the past, interpreters were simple; they read your source code at runtime, line by line, and then converted it to binary code, which then got executed. This all happened in real-time. The problem with this approach is that it’s slow, too slow for modern times, where applications are doing much more work and have many more users running the program concurrently, especially on the web.

Modern interpreters, like the Python Interpreter, have a caching mechanism. Instead of constantly re-reading the source code and going through all the startup steps like parsing over and over again, the Python interpreter will transform your raw source code into “.pyc” files.

This is not machine code; it’s basically just an optimized version of your source that’s already gone through the initial startup steps, like parsing. It’s not yet readable by the operating system. Bytecode is designed to be readable by the Python interpreter. The interpreter still has to execute this bytecode at runtime, translating each bytecode into an instruction for the computer to execute.

When you “publish” your application code, that version of code won’t change until you publish an update, so the parsing (and other code processing stages) only needs to happen once, and thereafter the interpreter knows that your code is valid and has passed all the checks, thus it doesn’t need to keep redoing all these steps.

While interpreters are slower than compilers, they do offer the benefit of hot-swapping code. In a compiled program, you have to recompile the entire program whenever you make a change and then redeploy that binary.

With interpreters like the Python interpreter, you can simply change the file(s) that have been updated, and the interpreter will automatically recreate “.pyc” files only for what has changed. This makes it easier to debug code or deploy portions of your code while the program is still running (you may still need to restart the application for all parts of your program to see the update).

So in summary, a “translator” program (like compilers or interpreters) takes source code (structured human-readable instructions for the computer) that you type in a code editor like VSCode and converts that to machine code. Machine code is binary instructions (1s and 0s) that the computer's processor can understand and execute.

What are Bytes?

Each bit (binary digit) does one very simple job: either on (1) or off (0), letting current through or not. Alone, this doesn’t amount to much, but together in a large program with millions of 1s and 0s arranged in a certain way, that program can achieve quite complex results like playing a movie or sending an email, and so on.

Now, imagine you own an electronics company, and your company sells TVs:

• The customer buys a finished TV, not the list of internal components or the names of engineers who assembled it.

• They don’t care which team achieved what or was responsible for which component.

• What they see is the final product: the TV. They don’t even see the components inside the TV; they just see the TV as a whole.

💡 Abstraction is about hiding complexity and exposing only what’s necessary.

The purpose of the above example is to make you understand what “Abstraction” is, because from here on out, when we move away from simple 1s and 0s, we start to get into the software realm, where programmers use varying levels of abstractions to ultimately achieve some result.

At the highest level, you write a program in a language of your choice, let’s say Python. This program consists of instructions written in human-readable code. Example:

print("Hello world!")

Eventually, the translator program will convert this into machine code (just made up for illustration purposes):

10110000 01100001
10110001 01100010

When looking at the machine code closely, you’ll notice that each block of numbers is exactly 8 digits long. This is what a “Byte” is: it’s a grouping of binary numbers in sequences of eight. Each Byte is, therefore, a “box” of sorts; it stores digital information by arranging 1s and 0s in different patterns (maximum 8 bits).

In the case of print(“Hello world!”) . Each letter or symbol is represented by a byte, which could look like:

p   -> 112 -> 01110000
r   -> 114 -> 01110010
i   -> 105 -> 01101001
n   -> 110 -> 01101110
t   -> 116 -> 01110100
(   -> 40  -> 00101000
"   -> 34  -> 00100010
H   -> 72  -> 01001000
e   -> 101 -> 01100101
l   -> 108 -> 01101100
l   -> 108 -> 01101100
o   -> 111 -> 01101111
    -> 32  -> 00100000
w   -> 119 -> 01110111
o   -> 111 -> 01101111
r   -> 114 -> 01110010
l   -> 108 -> 01101100
d   -> 100 -> 01100100
!   -> 33  -> 00100001
"   -> 34  -> 00100010
)   -> 41  -> 00101001

As you can see by arranging the 1s and 0s in different sequences, we can represent letters, numbers, and symbols.

If you look closely, we also have numbers 112, 114, 105, and so on. This is another significant thing to understand about bytes. A byte can only store numbers; you cannot put “p” into a Byte because computers only understand numbers. Thus, we assign a number to each character (this is known as character encoding, more on that later).

When you arrange the 1s and 0s together in a group of eight, these patterns have a maximum of 256 different combinations. Thus, a Byte can hold a number from 0 to 255.

In the real world, we use “Grams” to weigh things. Often in the real world, Grams numerically become too large and cumbersome to work with. It’s far easier to write and read 10 kg versus 10,000 grams.

Similarly, in the computing world, most programs will easily exceed 1000 Bytes. So a number like 1,073,152 Bytes can become really cumbersome to comprehend as a human; instead, we also have bigger units of measure like Kilobytes, Megabytes, Gigabytes, and so on.

These measures work as follows:

• 1KB = 1024 Bytes

• 1MB = 1024 Kilobytes

• 1GB = 1024 MB

Note: Due to marketing and varying hardware manufacturers, some use units of thousands, i.e., 1000 KB, 1000 MB, instead of 1024.

In summary, a “Byte” is a virtual unit of measure that is standardized across all computing systems. It groups together 8 bits (8 1s and 0s), and can be thought of as a “box” that can hold a number from 0 to 255.

What is character encoding?

On a mobile device, using icons and GIFs is common to convey certain meanings. We do this because, well, we’re chilling and lazy to type 🙃. Why type out a whole phrase, “I am so happy right now!” when you can just send a smiley face emoji?

Most people worldwide, regardless of what language they speak, can understand what an emoji means because at some point, we all agreed that 😃 means happy and ☹️means sad. For each of these emojis, we associated an emotion and globally accepted this meaning.

In computers, we know that there are just 2 digits that represent everything, either 1 or 0. When we group these two digits together, we can attach a more complex meaning to a Byte of data.

Still, a Byte can only represent numbers and the maximum number of 255. So, how on earth do we represent letters, let alone emojis?

To solve this problem, computer scientists developed character encodings. Character encodings basically associate a letter or character with a particular byte or bytes. Thus, character encodings are simply a standard of meaning for converting characters into bytes.

One of the earliest character encodings that worked well with the English language was known as “ASCII”. Similar to the emoji analogy, in ASCII, each character was associated with a particular number from 0 to 127, allowing the computer to capture the meaning of each character in a numerical format.

💡 This version of ASCII we’re talking about is know as “Standard ASCII”. later versions of ASCII can also use up more than 127.

Remembering our discussion on Bytes, you’ll recall that a Byte can store any number from 0 to 255. So, essentially, each ASCII encoding can be represented by a single Byte. Furthermore, each ASCII character only needs 7 bits out of the potential 8 bits in a byte (so at least one bit will always be off ‘0’).

💡 Notice how everything starts at 0. This is vital for later on in your career, because this pattern persists throughout programming. Especially in data structures like Arrays, in most languages these always start at zero.

While still used today in many systems, ASCII became very limited later when we needed to store more complex characters, such as emojis and characters from other languages, like Japanese. To solve this problem, a more robust encoding system, known as Unicode, was introduced.

Unicode is a tad bit more complex, like ASCII, it allows us to represent characters in a numerical form known as “code points”; however, Unicode itself is just a standard (Like a piece of paper with a list of rules to follow). You still need a system to actually store those code points inside Bytes.

These systems are known as encoding schemas, and there are a few varying implementations, like UTF-8, UTF-16, or UTF-32.

UTF-8 is the most popular; it basically uses up to 4 bytes to store characters. A simple English character would use 1 byte; however, some characters from a more complex language like Japanese can use 2-4 Bytes.

Encodings don’t only apply to text or characters; you also get encoding systems like the RGB system, which basically represents a colour using the 0-255 range. RGB uses 3 bytes to do this; each byte provides a value from 0 to 255 for the red, green, and blue color channels, respectively. By combining these three primary colors at different intensities, RGB encoding can represent approximately millions of different colours.

Some RGB examples:

rgb(255, 0, 0) ~ Red
rgb(0, 0, 255) ~ Blue
rgb(0, 255, 0) ~ Green
rgb(238, 130, 238) ~ Pink

The Math of Binary

In regular math, we are accustomed to the base 10 system, 1-10-100. Since we have a maximum of 10 digits(0-9), every time we reach the number 9, we need to “reset,” which means we add a 0 to the right, and the number on the left increases by 1.

9
10
11
12
13
15
15
16
17
18
19
20
21

Now, when we get to 9, we have run out of numbers. We can’t add 1 to the left because “9” is the highest digit in the numbering system. When this happens, we simply replace each “9” with a “0” and then add 1 to the far left, i.e., 10; the same applies to 99, 999, and so on.

9 + 1 = 10       = 10¹
99 + 1 = 100     = 10²
999 + 1 = 1000   = 10³
9999 + 1 = 10000 = 10⁴

Essentially, we are always working from right to left; we check the rightmost number to see if it’s 9, and if so, we apply the rules as per above. This is known as positional notation.

In base-10, when you reach a 9, you need to reset by adding a zero. In binary, it’s similar except you just add 1 or 0 to the left, so you effectively just increase the width of bits.

So to convert a binary number into a base 10 number, we use this width to determine how far this bit appears from the right (so effectively it’s position starting from 0, incrementing by 1 the further you go left).

Once you determine the position, this is now the power you must raise by a base of 2 to calculate the corresponding base-10 value.

In the case of 1 byte, the 8th number will be 2^0 (remember we always start with a zero), the 7th number 2^1, the 6th number 2². The further left you go, the higher the power. In the case of a Byte, the maximum power we can raise to is “7”, which is basically: 2×2×2×2×2×2×2=128

2^0 × 1 = 1
2^1 × 1 = 2
2^2 × 1 = 4
2^3 × 1 = 8
2^4 × 1 = 16
2^5 × 1 = 32
2^6 × 1 = 64
2^7 × 1 = 128
--------------------
Total     = 255

The above is a calculation based on a single Byte, where all values are “1” 11111111 This is the maximum value a Byte can hold, which, as you can see, adds up to 255.

What is a variable?

Since compilers and interpreters take care of converting code into bytes, we don’t often need to worry about how the code is translated and mapped on a character level. Instead, programming languages introduce their own “box” to put stuff in, called a “variable”.

A variable is essentially x number of Bytes that are grouped together to hold a certain portion of information, like a word or number, or even an image. I say “x” because, unlike a Byte, which has a fixed size of 8 bits, variables can be made up of any number of Bytes.

The number 52 For example can fit in a single Byte. Whereas an image would take up hundreds, if not thousands, of Bytes. Regardless of the size of either of these two entities, we can still store them in a single variable:

the_number_fifty_two = 52
profile_pic = '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--!Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2025 Fonticons, Inc.--><path d="M224 256A128 128 0 1 0 224 0a128 128 0 1 0 0 256zm-45.7 48C79.8 304 0 383.8 0 482.3C0 498.7 13.3 512 29.7 512l388.6 0c16.4 0 29.7-13.3 29.7-29.7C448 383.8 368.2 304 269.7 304l-91.4 0z"/></svg>'

In addition to a variable being able to store varied information, the box or size of the variable can also grow or shrink as you add or remove data. Example: when the variable is first created, we only have 2 names:

names = ["Kevin", "John"]

But, after we run the program, some process appends more names to the list, and thus our variable automatically grows in memory to accommodate more items:

.... do some computation to change names ....
names = ["Kevin", "John", "Peter", "Paul", "David", "Stacey"]

Languages like Python allow for this automatically without much effort from the programmer; however, other languages like C require you to specify the size of a variable upfront more often than not.

Most languages, like Python, have something called “Garbage collection”. Now, what this basically means is that the programming language will watch your variables and automatically delete the memory they occupy whenever that variable is no longer needed (known as going out of scope).

Garbage collection is often not available in some languages like C and C++; thus, as a programmer, you have to manually delete variables (or deallocate memory) when they are no longer needed and carefully create variables cognizant of the fact that you must take up as little memory as possible.

This is also why languages like C are used in game development, hardware drivers, and so on, because by manually managing memory, you can optimize your program to use as little RAM as possible and also deallocate memory as quickly as possible.

Since the garbage collection process itself eats up some memory and may not clear memory quickly enough, languages like Python are often used in web development and other environments where the code runs on bulky servers with lots of RAM, thus the need to minimize memory usage isn’t as important.

In summary, a variable is a box in memory that we use in programming languages to store information. Variables can store a wide variety of information, and each box can be of varying sizes depending on the type of variable.

Ultimately, each variable is just a grouping of bytes in memory. The variable can be made up of any number of Bytes (very similar to how we put 8 bits together to make one Byte, we put x number of Bytes together to make a variable).

Primitive Data Types

Okay, so we learnt that in programming languages we use a variable, which essentially is a box to put data in. We also know that variables can hold a wide variety of information.

The problem is that programming is precise; you want to be as precise as possible. You don’t want to mix numbers and letters willy-nilly. Some languages, like PHP and Python, are a bit flexible in this regard; they are what we call dynamically typed languages.

In Python, this is valid (although never a good idea):

my_name = "Billy"
my_name = 123

Now, looking at the code carefully. I created a variable “my_name” and then put the word “Billy” inside that box, then in the very next line, I replaced “Billy” with 123. Python allows this because it doesn’t enforce types by default.

Can you imagine working on a large financial system, and somebody mistakenly assigns text to a variable:

account_balance=500
account_balance="500"

The problem is that they may look like the same value, 500, but actually, they are not. See what happens when I try to update the account balance:

>>> account_balance="500"
>>> account_balance+1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only concatenate str (not "int") to str
>>>

Since “500” in quotes is actually a string (a chain of characters), Python throws an error. Saying: Wait a minute! I cannot add 1 to the word “500”. “500’ is not a number!

In some languages like Golang, TypeScript, C#, etc, doing something like the above will just not compile. Here’s a simple Go example:

package main

import "fmt"

func main() {
    var accountBalance int = 1
    fmt.Println(accountBalance)
}

Don’t worry so much about the syntax, such as “package”, “import”, “func”, etc I want you to zone in on this line:

var accountBalance int = 1

💡 The “var” keyword simply tells the Go compiler that you are making a new variable named “accountBalance”.

You will notice the int keyword. This is known as a primitive data type. The int data type tells the Golang compiler that this box in memory can only hold numbers, and more specifically, an integer. So when I do something like this, it will fail to compile:

package main

import "fmt"

func main() {
    var accountBalance int = 1
    accountBalance = "pokemon"
    fmt.Println(accountBalance)
}

The error:

➜  test go run main.go
# command-line-arguments
./main.go:7:19: cannot use "pokemon" (untyped string constant) as int value in assignment

Languages like Golang are known as “statically typed” languages.

Essentially, data types allow us to categorize variables so that we can store information in a structured way. The goal is to separate information based on different characteristics, such as integers (whole numbers), strings (text), floating-point numbers (decimals), and other distinct types.

💡 In the real world, you can think of data types as classifying different pieces of clothing. Just as we categorize items as "boots," "hats," or "shorts" based on their distinct characteristics and purposes, data types help us organize different kinds of information based on their unique properties and how they can be used.

Different languages have slight variations of these primitive data types, but in general, they mostly fall into the following categories:

• Char - a single character like ‘A’.

• String - a string of characters like “Hello”.

• Byte - we already know this one.

• Boolean - true or false

• Decimal number - so basically similar to money, e.g., 120.99

• Float - similar to Decimals, except they are not as precise

• Integers - these are whole numbers 1,2,3,4,5, and can be negative as well, 1,-2,-3. You also get variations in size, int, int64, and so on.

There’s way more to primitive data types than this simple article can cover, but hopefully, this is enough of a starting point for you to build upon.

Collections and dictionaries

Okay, so we learnt about variables, which are boxes in memory that you can put information into, and we learnt that each variable can have a type as well, like integers, strings, booleans, etc (primitive types).

In addition to these primitive types, variables can also hold more complex data types such as objects, lists, dictionaries, tuples, and more. Usually, complex data types are made up of one or more of the primitive types.

Let’s look at 2 of the most common types, i.e., lists and dictionaries. A list, as the name suggests, is a type that can hold a list of items:

shopping_list = ["eggs", "milk", "butter"]

Using Python, we created a “shopping_list” variable, which is a list of 3 items. Each list item in this case is of a primitive string type; however, the type can be just about any of the other types, including a complex type.

In programming, lists are very handy and used in just about every program. Some examples:

• In card games, you can use a list to store the deck of 52 cards.

• In e-commerce, A list can store all items in the user’s shopping basket.

• In a website’s navigation bar, a list can be used to store links to the relevant pages.

Unlike the primitive data types, a list can hold multiple values (usually these values are of the same primitive type, but not always), which makes it perfect for storing related items like products in a shopping cart.

In other languages, lists are also referred to as “arrays”; however, arrays can also have other features not supported by a simple list. So while the data type is similar, it’s not always exactly the same type.

For example, in JavaScript, arrays are the same as Python lists:

let shoppingArray = ["eggs", "milk", "butter"];

However, in PHP, while the type is called “array”, it’s much more powerful than a simple list since it also has features of the “dictionary” data type.

$shoppingArray = ["eggs", "milk", "butter"];
$personArray = ["name" => "Kevin", "surname" => "Naidoo"];

Lists or Arrays are usually zero-indexed, meaning that when we create the Python shopping list, under the hood, the language is assigning each item a position in the list:

💡 The # character in Python is a comment, meaning that the interpreter will ignore this. It’s merely there to help programmers communicate extra information to other programmers.

shopping_list = ["eggs", "milk", "butter"]

# Postions
# 0 => eggs
# 1 => milk
# 2 => butter

print(shopping_list[0])  # Will print eggs
print(shopping_list[1])  # Will print milk

The number we’re accessing above to print each item is known as an “index” and basically points to the position of the item in the list from left to right. Where the furthest position to the left is zero, and you keep adding 1 to the index as you move further right.

Moving backwards:

shopping_list = ["eggs", "milk", "butter"]

# Postions
# -3 => eggs
# -2 => milk
# -1 => butter

print(shopping_list[-1])  # Will print butter
print(shopping_list[-2])  # Will print milk

When moving backwards, the index starts from -1 and you add -1 to the index each time you move towards the left. The farthest item on the left when going backwards will be -3 In this case, it’s basically total_items * -1.

It’s not common to move backwards, so 99% of the time, you use this approach to simply get the last item when you don’t know the size of the list.

For example, in a deck of 52 cards, you may play some cards into a discard pile. Now, to access the last card in the deck, you can’t simply use cards[51]. Unless you count the cards in the deck again, you have no idea how many cards remain in the main deck.

By using cards[-1] This will always point to the last item in the deck so long as you have at least one card. Thus, you can dynamically always get the last item without having to constantly recount the deck.

Okay, so now we know that a list contains multiple items and each item is assigned a numerical position starting from 0. This data type is great if you don’t want to label the items, but what if you need to label the items?

For example, in your app, you have a dropdown asking users to select their country. You don’t want to manually type out each option by hand, so behind the scenes, you use a Python list populated from a database table to build an HTML dropdown:

# This data will be filled from the DB, just showing two values as an example.
countries = ["za", "us"]

<select>
     <option value="za">za</option>
      <option value="us">us</option>
</select>

The above is a simple HTML dropdown that will allow the user to select their country preference. The problem is, the user might not know their country code, and it’s far easier to select the full country name instead.

In the case of our dropdown, the server gets the value from the value field; it does not care about the label you see inside the <option> tags. In our Python code, we can therefore change the list into a dictionary (also known as a map in other languages). By doing so, we can set a custom index for each item, which will become the value set for the value field in our HTML.

Here’s how to make a dictionary:

# You can also put these on one line like a list, I did it this way for readability.
countries = {
   "za": "South Africa",
   "us": "United States",
   "uk":"United Kingdom"
}

A dictionary is very similar to a list, but as you can see, instead of 0,1,2,3 for the index, we set a custom index for each item, which is known as a key. The key will be of type string and serves as a label to better identify that piece of data.

Now, we can modify our dropdown to use the dictionary, so the result should be:

<select>
     <option value="za">South Africa</option>
      <option value="us">United States</option>
       <option value="uk">United Kingdom</option>
</select>

A dictionary, therefore, is much more powerful; not only can you store related information together, but you can also label data and build many interesting entities, like, for example, if you want to store information about a book: “The Pragmatic Programmer”.

the_book = [
   "The Pragmatic Programmer",
   "Journeyman to Master",
    1999, 
    "Pearson Education",
    352,
    9780132119177
]

We have 6 items, it’s kind of easy to guess what they are, but imagine if we had 20 items? This becomes really cumbersome. Instead, we can use a dictionary:

the_book = {
   "title": "The Pragmatic Programmer",
   "subtitle": "Journeyman to Master",
    "Year Published": 1999, 
    "Publisher": "Pearson Education",
    "Number of pages": 352,
    "Barcode / ISBN": 9780132119177
}

Looking at the second item, it’s far easier to identify what each data point means just by using the dictionary’s key. When we want to print an item, we can do so in a similar way to that of lists, by using the key instead of a numerical position:

print(the_book['title']) # The Pragmatic Programmer
print(the_book['Year Published']) # 1999

Conclusion

In this article, we learned about one of the most fundamental concepts in computer programming, i.e., data. We started at the very heart of the computer to understand how binary numbers represent electrical signals in transistors that allow the current to flow or prevent the current from flowing in a particular direction.

We then built upon this knowledge to learn about Bytes , which is a fundamental “Lego block” of modern computing. We even learnt some math, and further expanded into programmatic concepts like compilers, variables, data types, and more.

We’ve seen a little sneak peek into Python, Golang, and other languages.

All in all, this was a fun article to write; it allowed me to refresh some of these concepts I learned many, many years ago. I hope that this was an ideal primer for you, to introduce you to these important fundamentals and help you grow as a developer.

Use this as a springboard into the world of programming; I merely got your feet wet, but to become a good programmer and have a long-lasting career in the industry, you must keep learning and constantly dig deeper.

Happy coding! I’m planning on more content like this that dives deeper into the engineering aspects of programming, so please join the newsletter or check back soon.

Thank you so much for reading 🙏

Cheers 👋

In this article, let’s look at a quick-fire way of implementing an auth system in a matter of minutes.

A quick Django primer

This article is designed for developers who are familiar with Django, but just in case you are a complete beginner or just “rusty,” let’s go through some basics:

Virtual Environment:

To get started with Django development, you need a virtual environment. A virtual environment is simply an isolated Python installation that won’t interfere with the global Python installed on your system.

Virtual environments are especially useful on Linux machines. Since most Linux distros use Python for various core libraries, by installing your project packages at the system level, you could potentially break your whole desktop environment!

To set up a virtual environment for our project, run this in your terminal:

mkdir django_projects
cd django_projects
python -m venv .venv
source .venv/bin/activate

To exit the virtual environment, just type “deactivate” in your terminal, and to go back, simply run the last command as per above, ensuring you’re in the correct folder.

Now let’s install Django and create our project:

pip install django
django-admin startproject authsystem
cd authsystem

When you create a new Django project, you should see this folder structure:

.
├── authsystem
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── manage.py

Very barebone and minimal, such is the beauty of Django! Inside your project, you will find a “manage.py” command, similar to the first “django-admin” command. This will allow you to perform various console tasks. The only difference is that “manage.py” is scoped to your project and will be aware of your project’s settings and apps.

Let’s create an application called “accounts”

python manage.py startapp accounts

Django has a very clean architecture in that you have projects and apps:

• Projects: Are a collection of apps, usually your website project, or a library or framework that you're developing. They help organize related code and resources under a single umbrella, making it easier to manage dependencies, configuration, and deployments.

• Apps: Are like small libraries, they should have one major purpose. Example: an "accounts" app manages everything related to user accounts, a “documents” app manages everything related to parsing and storing documents and so forth.

An app generally consists of:

├── __init__.py
├── admin.py
├── apps.py
├── migrations
│   └── __init__.py
├── models.py
├── tests.py
└── views.py

• admin.py: We won’t use the admin panel in this tutorial, but basically registering models here will create a CRUD interface in the Django admin panel for that model.

• migrations: Any logic that needs to alter your database structure.

• models.py: A local mapping of database tables using Django’s powerful ORM.

• views.py: These are actually “controllers” when thinking in “MVC” terms. The “V,” by the way, in Django is actually the controller, and “T” is the templates, thus, Django is an “MVT” framework.

Django will not automatically register “apps” you generate; you’ll need to register the app manually in order to use this app within the project. Luckily, this is fairly easy to do, just add “accounts” to your settings.py file as follows:

# Update authsystem/settings.py

INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "accounts"            # <---- the app we created.
]

While we’re in the settings file, let’s tell Django where to find our HTML templates.

First, let’s create the template folder. You should be at the project root at the same level as “authsystem” and “accounts”:

mkdir templates

Next, let’s update our settings file:

TEMPLATES = [
    {
        "BACKEND": "django.template.backends.django.DjangoTemplates",
        "DIRS": [f"{BASE_DIR}/templates"], # <---- Add template folder here.
        "APP_DIRS": True,
        "OPTIONS": {
            "context_processors": [
                "django.template.context_processors.debug",
                "django.template.context_processors.request",
                "django.contrib.auth.context_processors.auth",
                "django.contrib.messages.context_processors.messages",
            ],
        },
    },
]

Django is very flexible; you can place templates just about anywhere, including inside each individual app’s folder. Since the goal of this project is to be a single webapp, we are not building a library or shareable components, thus we’ll just use one templates folder at the project root.

Getting started

Great! Now that you have a project and an accounts app. Let’s first run migrations:

python manage.py migrate

By default, Django will use “sqlite” as its database, which makes local development much easier, as you don’t have to configure anything in order to get a database up and running. Since Django is ORM-based, it’s usually easy to just swap out the database type in production to something like PostgreSQL.

You will notice a whole bunch of migrations are actually run, even though we’ve not added a single model or database table???

  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying sessions.0001_initial... OK

These are migrations for some Django admin features, including various tables for auth, such as permissions, groups, and a user table to store registered users. Sensible defaults you’ll use in nearly every project, so it’s a welcome addition almost always.

To get started with our custom login system, let’s add a few routes to authsystem/urls.py as follows:

💡 A cleaner way to add urls, is to use the “accounts” urls.py file and then import the “urlpatterns” in the main urls.py. I’m putting everything in the project’s urls.py for simplicity.

from django.contrib import admin
from django.urls import path

from accounts.views import account_login, account_logout, account_register,dashboard

urlpatterns = [
    path("login/", account_login, name="login"),
    path("register/", account_register, name="register"),
    path("logout/", account_logout, name="logout"),
    path("dashboard/", dashboard, name="dashboard"),
]

Your IDE or code editor might complain that these functions don’t exist, just ignore those for now. We’ll add them in a bit.

Building the login page

Before we get to the actual view functions, let me just list out all the imports we're going to need here so it’s easier for you to follow along instead of constantly repeating these:

from django.contrib import messages
from django.contrib.auth import authenticate, login, logout
from django.contrib.auth.decorators import login_required
from django.contrib.auth.models import User
from django.shortcuts import redirect, render

from .forms import RegisterForm

Now that we have the routes in place, let’s build the login view. In your “accounts/views.py” file, add the following:

def account_login(request):
    if request.user.is_authenticated:
        return redirect("dashboard")

    if request.method == "POST":
        username = request.POST.get("username")
        password = request.POST.get("password")
        user = authenticate(request, username=username, password=password)

        if user is not None:
            login(request, user)
            return redirect("dashboard")
        else:
            messages.error(request, "Invalid username or password.")

    return render(request, "accounts/login.html")

ℹ️ Notice I am prefixing some of these views with “account_”. This is because we import some functions from Django named “login”, “logout” and so on, thus, doing so just prevents clashing with those function names.

We first use request.user_is_authenticated to check if the user is logged in, and if they are, just redirect to the user dashboard page. Otherwise, we take 1 of 2 actions:

• If the request is a POST request, it means that the user filled in their username and password. We should therefore validate those credentials. We use the auth helper function provided by Django authenticate , which will take care of hashing the password and comparing it with the password stored in the DB.

• If it’s a GET request, just show the login form.

Let’s create a folder to store all of our auth templates:

mkdir -p templates/accounts
touch templates/accounts/login.html
touch templates/accounts/register.html
touch templates/accounts/dashboard.html

Next, paste this into the “login.html” template

{% if messages %}
<div class="mb-4">
    {% for message in messages %}
    <div>
        {{ message }}
    </div>
    {% endfor %}
</div>
{% endif %}

<form action="{% url 'login' %}" method="POST">
{% csrf_token %}

<label>Email Address</label> <br />
<input type="text" id="username" name="username" required /> <br />  

<label>Password</label> <br />
<input type="password" id="password" name="password" required /> <br />  
<input type="submit" value="Login" />
</form>

This is a very ugly, unstyled form, but I kept it simple so you can see what’s going on. Notice we also print messages; this is another useful Django feature. It’s basically a flash message where you can pass success/failure messages between views and between the view and the template.

Handy for showing error messages when the user doesn’t fill in something or fills in incorrect information.

Also note: we include {% csrf_token %} which is a security mechanism to protect your forms from “Cross-Site Request Forgery” attacks. Since this is a simple 2-field form, I didn’t create a form in “forms.py”, but usually, with Django, you would create a form class when dealing with user input.

Building the registration view

For this one, we are capturing quite a few fields, therefore, we’ll use Django forms as well to manage the form handling better. Let’s start off by creating a “forms.py” in the “accounts” folder:

touch accounts/forms.py

from django import forms
from django.contrib.auth.forms import UserCreationForm
from django.contrib.auth.models import User
from django.core.exceptions import ValidationError

class RegisterForm(UserCreationForm):
    email = forms.EmailField(
        required=True,
        widget=forms.EmailInput()
    )
    first_name = forms.CharField(
        max_length=50,
        required=True,
        widget=forms.TextInput()
    )
    last_name = forms.CharField(
        max_length=50,
        required=True,
        widget=forms.TextInput()
    )

    class Meta:
        model = User
        fields = ('first_name', 'last_name', 'email', 'password1', 'password2')

    def save(self, commit=True):
        user = super().save(commit=False)
        user.username = self.cleaned_data.get('email')
        user.email = self.cleaned_data.get('email')
        user.first_name = self.cleaned_data.get('first_name')
        user.last_name = self.cleaned_data.get('last_name')

        if commit:
            user.save()
        return user

Django provides a UserCreationForm class that will take care of validation and hashing of the password, so we don’t have to reinvent the wheel here. We override the save method to ensure that the username is set to the user’s email address. This “username” field comes from an old school way of doing things when usernames were a thing; nowadays, most just use the email as a username.

Also, overriding save allows you to do more customizations, such as:

• Setting a User profile. We’re using the default Django user model, but your application may need to store additional information such as birthdate, cellphone number, address, etc… In such a case, simply create a new model “UserProfile” and use the: models.OneToOneField relationship to link these models. Thereafter, you can store whatever extra information is needed via the save method.

• Link a user to a Team. Great for SaaS type applications.

⚠️ It’s worth mentioning that Django’s username field has a max-length of 150 characters. This should be generally fine for most email addresses, however, you can always alter the column length using a migration. To get an idea on how, have a look in: .venv/lib/python3.12/site-packages/django/contrib/auth/migrations

Next, let’s build our register view, in views.py:

def account_register(request):
    if request.user.is_authenticated:
        return redirect("dashboard")

    if request.method == "POST":
        form = RegisterForm(request.POST)
        if form.is_valid():
            try:
                user = form.save()
                messages.success(
                    request, "Account created successfully. You can now log in."
                )
                return redirect("login")
            except Exception as e:
                messages.error(request, f"Error creating account: {e}")
    else:
        form = RegisterForm()

    return render(request, "accounts/register.html", {"form": form})

This is very similar to the login view, except we use a form this time. Finally, you need to paste this HTML in templates/accounts/register.html

<div>
  <h2>Register</h2>

  {% if messages %}
  <div>
    {% for message in messages %}
    <div>
      {{ message }}
    </div>
    {% endfor %}
  </div>
  {% endif %}

  <form  action="{% url 'register' %}" method="post" />
    {% csrf_token %}

    <div>
      <label for="{{ form.first_name.id_for_label }}">First Name</label>
      {{ form.first_name }}
      {% if form.first_name.errors %}
        <div>{{ form.first_name.errors }}</div>
      {% endif %}
    </div>

    <div>
      <label for="{{ form.last_name.id_for_label }}">Last Name</label>
      {{ form.last_name }}
      {% if form.last_name.errors %}
        <div>{{ form.last_name.errors }}</div>
      {% endif %}
    </div>

    <div>
      <label for="{{ form.email.id_for_label }}">Email</label>
      {{ form.email }}
      {% if form.email.errors %}
        <div>{{ form.email.errors }}</div>
      {% endif %}
    </div>

    <div>
      <label for="{{ form.password1.id_for_label }}">Password</label>
      {{ form.password1 }}
      {% if form.password1.errors %}
        <div>{{ form.password1.errors }}</div>
      {% endif %}
    </div>

    <div>
      <label for="{{ form.password2.id_for_label }}">Confirm Password</label>
      {{ form.password2 }}
      {% if form.password2.errors %}
        <div>{{ form.password2.errors }}</div>
      {% endif %}
    </div>

    <button type="submit">Register</button>
  </form>

  <div>
    <p>Already have an account? <a href="{% url 'login' %}">Log in</a></p>
  </div>
</div>

The Logout view

Logout is the simplest view of the 3, you simply just destroy the session and redirect to the login page:

def account_logout(request):
    logout(request)
    return redirect("login")

Restricting access

Now that you have a basic login system in place. We need a place for the user to be redirected to after login. Let’s create the dashboard view as follows in accounts/views.py

def dashboard(request):
    return render(request, "accounts/dashboard.html", {})

And some simple HTML in templates/accounts/dashboard.html:

<h1> Welcome {{request.user.first_name}} </h1>

One problem, though, if you visit /dashboard without logging in, you can still access this page! To restrict this page such that only logged-in users can access it, we simply need to import and use the @login_required decorator as follows:

... other imports here ....
from django.contrib.auth.decorators import login_required

@login_required(login_url="/login")
def dashboard(request):
   return render(request, "accounts/dashboard.html", {})

Now, you should get redirected to the login page. Easy right?

Conclusion

The Django auth system is really powerful and robust. This article was merely a gentle introduction, there’s so much more you can do, especially with authorization, decorators, and more.

Hopefully, this gives you a good enough start to build out your own auth system. Even though in the modern era, you probably could get some sort of boilerplate or use a 3rdy party auth service, it’s still very handy to understand the fundamentals of how auth systems work.

Both hold some truth, but there's no need for concern. I am not an AI expert, but for the last few years, I have worked with almost every model that exists from GPT4o to Sonnet 3.5 to even DeepSeek R1.

I have built complex workflows, trained classifiers, built simple chatbots, voice AI solutions, and much more. Throughout this experience, I have learned about the limitations of AI and the difference between actual functionality versus marketing hype.

In this article, we'll go on a mini journey, inside the deep underbelly of the AI world and explore where the technology is today and how it'll impact you as a developer in the coming years. And yes, we'll also build an Agent from scratch in Python.

💡 Are you new to Python? Learn Python with LearnPython.com courses. Fun interactive courses that are based on real-life business scenarios, meaning you’ll be writing Python code and seeing the results instantly. No need to install Python or other tools on your device, everything happens through your favorite web browser (Sponsored content).

What is an AI agent anyway?

This term is thrown around all the time as if it's something out of this world. No, it ain't! It's just a program, just functions a tad bit differently.

Agents are just orchestration programs that can parse text (or other media formats), analyze the contextual meaning of that text using LLMs, and then execute tasks based on whatever requirement is set out in that text.

To clarify, take a spec for building an exchange rate calculator. The ticket might look like:

Design a program that will take in a dollar amount 
and generate the relevant Rand or 
Euro amount based on the current exchange rate.

As a programmer, you now will build an interface with an input box. That input box will have some validation so that it ensures the end user only enters a valid number.

If they try something like this: "What is $500 in Euros?"

The validation will kick in and alert the user with a message: "Sorry, your input is invalid. Please only enter numbers"

As you can see, a conventional program is more strict where types are enforced and users must input data in a structured way. Agents on the other hand don't have this limitation, you simply can chat with the agent and it'll parse the natural language into something that the computer can understand.

Instead of the programmer explicitly writing every single pathway a program can take, agents are more flexible in that they do have some level of decision-making, enabling them to dynamically build workflows on the fly.

In our currency conversion program, the programmer would have to build some sort of API integration. When the user clicks calculate, behind the scenes the numerical value entered is sent to a currency conversion API, and then the API responds with the relevant conversion.

The programmer needs to cater to every step in this workflow because he/she needs to take that numerical value and build some kind of JSON object. Thereafter, provide some sort of authentication, followed by making a post request, then getting back the post data, parsing it and finally displaying the final result.

In the case of the Agent, we don't need to explicitly write this workflow. The agent can build the JSON object all on its own and read auth credentials from your ENV because it's able to leverage the power of modern LLM coding abilities.

To give you another practical example:

In the above diagram, the agent first asks the LLM for a plan of action, next, it works through that plan step-by-step. At each step, it'll go back to the LLM for any additional information or it might execute some external task like querying an API.

In the case of this diagram, the user asks the agent to fetch the price of a particular Amazon listing. The Agent, in turn, queries the LLM for a list of tasks to do which might look like these:

• Spin up a Python environment.

• Write a script to visit the product page.

• Wait for the DOM to load and scrape the HTML.

• Use a regex or querySelector to find the DOM element with the price.

• Extract the price and parse it into a float.

• etc...

When the agent runs the Python script, it may crash. This now blocks the agent from executing other steps, so it goes back to the LLM and passes the errors to the LLM for further diagnosis. This ability to self-correct is both a blessing and a curse, more on that later...

The LLM then returns back the fixed code, which the agent executes and continues on with its tasks.

In summary; agents can generate code, query LLMs, and even talk to APIs, they are semi-autonomous programs that can function without explicit instructions for every single pathway. Still, as powerful as agents are, they still need you! The programmer.

Similar to how you needed your parents to provide you with a support system, education, clothing, food, and so forth... Agents need programmers to build the "Lego blocks" of functionality and define the rules of the environment in which it runs.

By "Lego blocks" I mean adapters, while an LLM could generate code for an API integration, this is probably not a good idea, because a) it can hallucinate and just error out half the time. and b) there's no guarantee we can trust the source API.

A better approach would be to build an adapter for the API of your choosing and just provide the agent with some information on how to use that adapter (LLMs usually refer to these as tools).

Agents are only as good as the best LLM

Take a set of twins, they are identical in most aspects including height and body weight, schooling, etc...

One drives a Toyota Corolla and the other a Ferrari. They both have similar driving skills, they swap cars from time to time and they love racing each other.

Regardless of which brother drives which car, the Ferrari always wins.

Why!!?

I won't go into the details, but clearly, the Ferrari has a much superior engine, hence why it can beat the Corolla any day without much fuss.

Coming back to agents. Agents do not have any reasoning capability of their own, they rely heavily on LLMs.

These LLMs have no measure of quality or completeness, they just generate tokens that look correct mostly but don't apply the same level of quality checking that a human would do nor do they have the same level of understanding as a human.

Even worse, as you keep prompting the model, it'll keep giving you some sort of response with confidence even if that response is just, well, plain garbage.

Throw in an agent and you got big trouble! Because the agent's execution plan is being built by the LLM. It's just going to loop through each step and just execute. Sure, agents can detect problems and try to re-ask the LLM to fix issues but since the LLM will most likely hallucinate at some point, the agent could actually get stuck in an endless loop or most likely will not complete tasks holistically.

This is why agents need to be programmed in a certain way with loads of guardrails. They are far from autonomous. They need constant fine-tuning, and can only be used for very niche tasks.

Programmers rule the world!

Does Zuck really want to promote agents to mid-level software engineers? Can you imagine the agent getting stuck in an endless loop, constantly adding and deleting lines of code for 4 hours 😂 and then submitting a PR with broken code?

Then another agent merges that code, and 💣 half of Facebook is down for hours! Hey! It's possible, looking at the crappy code that the so-called best LLMs generate, I won't be surprised.

No! This is just marketing gimmicks, yes of course a company like Facebook has some percentage of developers that perform very niche tasks.

These tasks can then be mostly automated with agents, and maybe one or two senior engineers who supervise and merge code, allowing Facebook to let go of some percentage of their workforce. This is highly possible and most likely is happening and will continue to happen.

For the rest of us, agents are going to bring on more work. Someone's going to have to build those APIs and write those guardrails to prevent agents from getting stuck in endless loops.

Furthermore, APIs and software are constantly changing thus one needs to constantly keep the agent code in sync with the rest of the software changes and of course make those software changes.

Besides, I'm pretty sure salespeople are just going to sell more stuff because developers can now automate more things, thus freeing up their time to work on more stuff.

Finally, no tech CTO or founder is just going to hand over their AWS keys to an agent and fire all their developers. That's insane, AI is simply not at that level and probably won't reach that level anytime soon.

PS: You should read my earlier article to learn more about "Tunnel Syndrome" here.

AI agents are the future just like the mobile phone

Remember the doomsayers who were saying computers are going to die off because of mobile phones?

We developers, are the computers in this era.

You'll find, while yes a certain group of computer users now opt for a tablet or a mobile phone instead of a laptop, the amount of laptop and desktop users is still pretty huge.

This is because mobile phones are mostly communication and consumption devices, they service a slightly different market and now developers have to make everything responsive so that we can support more devices.

Similar to now having to build two layouts, one for mobile and one for desktop, or adding a ton of media queries to make your website responsive. The workload for developers is just going to increase because now every company is going to want to deploy some AI feature or agent.

Soon, agents will be the new chatbot that everybody needs, so we'd be building agents.

Thus agents are here to stay and may replace some jobs but ultimately will create more jobs because we going to need more developers to build and maintain these agents.

Let's build an agent from scratch

Okay enough talk, let's roll up our sleeves and actually build an Agent. Before we get started, let me just stress this is a very basic example for educational purposes.

In the real world, you would want to error handle a lot better and probably break this up into multiple classes.

The agent:

from pydantic import BaseModel
from openai import OpenAI
import time

class ExecutionPlan(BaseModel):
    steps: list[str]

class AIAgent:
    def __init__(self, debug=False):
        self.client = OpenAI()
        self.debug = debug


    def landing_page(self, **kwargs):
        prompt = kwargs['prompt']
        goal = "build a modern landing page"
        execution_plan = self.build_execution_plan(goal, prompt)
        messages = [
            {"role": "system", "content": "You must build a landing page for this user based on their requirement."},
            {"role": "user", "content": prompt},
        ]

        for step in execution_plan.steps:
            if self.debug:
                print(f"Executing step: {step}.")
            messages.append({"role": "system", "content": step})
            response = self.ask_model(messages)
            if self.debug:
               print(f"Done: {response}")
            messages.append({"role": "assistant", "content": response})

        messages.append({"role": "user", "content": "Please respond with the final design for my landing page requirement. Please return only HTML and no extra commentary."})

        landing_page = self.ask_model(messages)

        with open("landing.html", "w+") as f:
            f.write(landing_page)

        return "Done. Please view landing.html in your browser."

    def invoke_action(self, action, **kwargs):
        if not hasattr(self, action):
            return "Sorry, I have no idea what to do with this request!?"

        getattr(self, action)(**kwargs)

    def ask_model(self, messages, model="gpt-4o-mini", temperature=0.7, response_format=None):
        i = 0
        while i < 3:
            i += 1
            try:
                if response_format is not None:
                    response = self.client.beta.chat.completions.parse(
                        model=model,
                        messages=messages,
                        temperature=temperature,
                        response_format=response_format
                    )
                    return response.choices[0].message.parsed
                else:
                    response = self.client.chat.completions.create(
                        model=model,
                        messages=messages,
                        temperature=temperature
                    )
                    return response.choices[0].message.content

            except Exception as ex:
                print(ex)
                time.sleep(2)

    def intent_router(self, user_prompt) -> ExecutionPlan:
        prompt = """
            You must analyze the following user prompt and determine which action below best describes
            the user's request. Respond only with the action e.g. [email]

            1) [email] - the user wishes to send an email.
            2) [landing_page] - the user wishes to build a landing page.
            3) [book_calendar_date] - the user wishes to book a slot in their calendar.
        """

        intent = self.ask_model([
            {"role": "system", "content": prompt},
            {"role": "user", "content": user_prompt}
        ])

        if self.debug:
            print(f"RAW intent: {intent}")

        return intent.replace("[", "").replace("]", "").strip()

    def build_execution_plan(self, goal, prompt):
        system_prompt = f"""
            Given the current goal:'{goal}' and the user prompt, return a step-by-step execution plan
            for an Agent to work through, so that it can adequately and efficiently fulfill the users request.
        """
        messages = [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": prompt}
        ]

        return self.ask_model(messages, response_format=ExecutionPlan)

    def run(self, prompt):
        intent = self.intent_router(prompt)
        if self.debug:
            print(f"Intent for: '{prompt}' === {intent}")

        return self.invoke_action(intent, prompt=prompt)

def main():
    agent = AIAgent(debug=True)
    result = agent.run("I would like to build a landing page for my plumbing business. Please include images for pixels.com and generate copy that makes sense for industry.")

    print(result)

if __name__ == "__main__":
    main()

Let's break this up step-by-step. The first thing you want to do in the agent process is determine what the user is actually trying to do, this is very similar to routing in a traditional web application where you map a URL route to a controller.

In the case of our Agent, we don't have URLs, instead the LLM parsers the text prompt and determines a keyword (i.e. the "intent") that best describes what they want to achieve:

    def intent_router(self, user_prompt) -> ExecutionPlan:
        prompt = """
            You must analyze the following user prompt and determine which action below best describes
            the user's request. Respond only with the action e.g. [email]

            1) [email] - the user wishes to send an email.
            2) [landing_page] - the user wishes to build a landing page.
            3) [book_calendar_date] - the user wishes to book a slot in their calendar.
        """

        intent = self.ask_model([
            {"role": "system", "content": prompt},
            {"role": "user", "content": user_prompt}
        ])

        if self.debug:
            print(f"RAW intent: {intent}")

        return intent.replace("[", "").replace("]", "").strip()

Once we know the intent, we can easily determine what method/action to execute:

    def run(self, prompt):
        intent = self.intent_router(prompt)
        if self.debug:
            print(f"Intent for: '{prompt}' === {intent}")

        return self.invoke_action(intent, prompt=prompt)

In our "run" method which is the entry point that starts up the agent, we use the intent to dynamically execute a method on the "AIAgent" class.

Taking a peek inside "invoke_action", you will see that it's basically just checking if the class has a method that's named exactly the same as the relevant intent.

    def invoke_action(self, action, **kwargs):
        if not hasattr(self, action):
            return "Sorry,...."

        getattr(self, action)(**kwargs)

If the method exists, we just execute it and we pass any extra arguments in as well via "kwargs".

In my example I am asking the agent to build a landing page, thus the intent is "[landing_page]" and if you look inside the "AIAgent" class you will notice a method with this same name:

    def landing_page(self, **kwargs):
        prompt = kwargs['prompt']
        goal = "build a modern landing page"
        execution_plan = self.build_execution_plan(goal, prompt)
        messages = [
            {"role": "system", "content": "You must build a landing page for this user based on their requirement."},
            {"role": "user", "content": prompt},
        ]

        for step in execution_plan.steps:
            if self.debug:
                print(f"Executing step: {step}.")
            messages.append({"role": "system", "content": step})
            response = self.ask_model(messages)
            if self.debug:
               print(f"Done: {response}")
            messages.append({"role": "assistant", "content": response})

        messages.append({"role": "user", "content": "Please respond with the final design for my landing page requirement. Please return only HTML and no extra commentary."})

        landing_page = self.ask_model(messages)

        with open("landing.html", "w+") as f:
            f.write(landing_page)

        return "Done. Please view landing.html in your browser."

The method starts off by defining a "goal", this is more descriptive than the single keyword intent and will be used to prompt the model again to get a detailed step-by-step workflow.

execution_plan = self.build_execution_plan(goal, prompt)

Looking inside the execution plan method, we use structured outputs so that we can tell the LLM to return a list of steps in the format of "ExecutionPlan" which is just a plain pydantic model:

class ExecutionPlan(BaseModel):
    steps: list[str]
...
    def build_execution_plan(self, goal, prompt):
        system_prompt = f"""
            Given the current goal:'{goal}' and the user prompt, return a step-by-step execution plan
            for an Agent to work through, so that it can adequately and efficiently fulfill the users request.
        """
        messages = [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": prompt}
        ]

        return self.ask_model(messages, response_format=ExecutionPlan)

    def run(self, prompt):
        intent = self.intent_router(prompt)
        if self.debug:
            print(f"Intent for: '{prompt}' === {intent}")

        return self.invoke_action(intent, prompt=prompt)

Finally, we just loop through all the steps and prompt the model one by one until the final landing page is built.

Conclusion: AI Agents are here to stay

Don't be fooled by these big-name CEOs and tech entrepreneurs who just spread propaganda so that they can drive more investment into their companies. They use AI agents as a means to stir up market interest and get non-technical people all excited.

This is an age-old marketing trick! To these people, they are obsessed with the bottom line. A sad side-effect of capitalism! Greed for growth at any cost is the endgame.

The reality is that the technology while useful, is just over-hyped. It's been a good few years since the first version of ChatGPT was released and yet AI products just fall into two categories: AI Wrappers or value adds.

At the end of the day, Agents, AI, and all these tools are just that, tools. They are here to stay and plenty useful, but do not have reasoning and thinking capabilities on the same level as that of a human being, and probably never will!

Why am I so confident?!

AI Tunnel Syndrome is one of AI's biggest weaknesses and it's not going to get better any time soon, regardless of "o3" or whatever model comes out next.

Preamble: The Consciousness

Okay, so "AI Tunnel Syndrome" is a term I just coined while thinking about AI stuff, but bear with me for now, we'll get back to what I mean in a minute.

The thing is, to be human, you are more than just a library of knowledge. You are filled with memories, emotions, and experiences. You are a being that can adapt to just about any environment and thrive even in the most impossible circumstances.

You have a worldview and consciousness.

AI, on the other hand, is frozen in time. You see, there is a reason why we call them "models". Each new iteration has a version number. Do you have a version number? No, because your knowledge and experience grow every second of every day.

Models have no consciousness, they are just algorithms that can parse, scan, and find patterns in a large corpus of information.

The version indicates that the model's cutoff knowledge base has just moved further down the line with more data and various improvements on both the dataset and the algorithm.

The LLM stops learning at this version cutoff; every conversation thereafter is just an interaction, and there's no impact on the model's persistent knowledge base (although you can supplement the AI’s knowledge base with RAG and tool calls).

The World View

As human beings, we understand not only the knowledge we accumulate over time but also how that knowledge fits into the broader context of the world around us.

While we are not soothsayers(well, most of us anyway), we do have the ability to see into the future to a certain extent and make decisions based on all three temporal dimensions: the past, the present, and the future.

To illustrate what I mean, here's an example:

Imagine it's a bright sunny day, incredibly hot, and you are sweating like crazy. At this moment, you have all the knowledge of everything you've ever learned. You have memories and past experiences too. Sure, you might forget stuff, but overall, you remember most things.

Your mind floats to thoughts of a nice cold ice cream, and you remember how refreshing and cooling it feels to have one from past experiences.

This now drives your decision to buy an ice cream! But wait! You bought an ice cream from the guy down the street 3 months ago, and it wasn't all that great.

You also know, it's fast approaching lunchtime, and at 12 PM, the city center gets really busy. On such a hot and sweaty day, the last thing you want to do is get stuck in a crowd of people.

So eventually, you decide to take a quiet side street instead, where you can buy your ice cream away from the city's hustle and bustle.

While enjoying that ice cream, you look to the city center and smile 😊, it's lunchtime and everyone is flocking there! Phew, dodged a bullet, didn't you?

Wait, what does this have to do with AI!?

Patience 🙏 eager-beaver, I'm getting there 🙃

In the above example, your brain is using senses, memories, past experiences, and emotions altogether, all at once to build this complex decision tree in record time, whilst still consuming very little energy.

Furthermore, you are even using that information to predict the future and other possible edge cases and consequences.

Models on the other hand, have no worldview and are narrow-focused; they take your prompt and do some math on it to determine patterns and complex relationships between words.

Thereafter, based on the data they've been trained on, they generate a response that's most likely going to satisfy the prompt. Depending on the quality of your prompt, this will impact the calculation performance and thus the final result.

This is perfectly fine for a simple "lookup" question like: "Explain the theorem of Pythagoras". The question has one clear goal, thus the model can easily determine the meaning and generate an accurate response.

Give it a prompt like "Build me a landing page for an electrician, the colors are red and black. For the content, use placeholder images and text. I need 5 pages: About, services, contact, gallery..."

If you ask a human to do this task, even with this poor specification, they probably will come back with a complete website. AI on the other hand, even powerful models like Sonnet will build you something, but it's never going to be polished and will most likely miss obvious features.

What is AI tunnel syndrome?

With your worldview in the case of the ice cream example, you were able to piece together a complete picture of the environment, the taste, the texture, the horror of overcrowding, and the pleasant memory of eating the ice cream.

Coupled with the fact that you are conscious, every event in your life no matter how small or big it is, contributes to your worldview, your memories, and your learning.

To drive home the true meaning of "AI Tunnel Syndrome", let's look at another practical example. In the real world just writing code is not enough; you need to sit in meetings with non-technical people; they often don't understand their actual requirements, budget constraints, and various factors that could come into play when building out their application requirements.

Can you imagine a request: "We need something like Twitter, so we can all communicate internally with each other, share files, and even video call. Please, can you build this for us?"

🤖: Sure, has access to the codebase, goes off, generates some Next.js code. Mostly works, but there is no auth integration, so just about anyone can initiate a chat. Also, there's no mime validation on files, so one can upload just about anything, and finally, the UI has a logo of the old Twitter bird icon?!

👨‍💻: I ain't got 3 months to build that, we'll just set up Slack.

The human immediately identifies information beyond the prompt, the context, and the consequences of executing the task concerning time and budget constraints.

The AI, on the other hand, finds the relevant information and starts building but doesn't cater to many edge cases or even think about just using a pre-built solution like Slack.

💡The lack of this complete "big picture" view (which is second nature to human beings) is what I call "AI Tunnel Syndrome".

Enter the Agent

Agents are an interesting next evolution of AI models. Think of an agent as an orchestra conductor. In an orchestra, you have several different instruments: violins, clarinets, trumpets, and so forth. Each instrument produces a different sound.

Can you imagine if each musician played from a different music sheet, the result would be chaotic! Instead of harmony, you'd hear a mix of competing melodies, rhythms, and keys that don't complement each other. Each of these instruments alone sounds great, but as soon as you put them together, you need some sort of structure.

The conductor ensures that they all stay in sync and play harmoniously, producing beautiful music.

Bringing it back to Agents. LLMs simply take input (whether it’s text, voice, images, etc) and respond with some output. They can look at any context data you may supply the prompt, like a PDF or previous chat messages, but beyond that, they cannot access information outside of the model.

To solve this problem, AI companies developed “tools”. With tools, you can connect the LLM to an external service like an API, search engine, emails, calendars, or any other data source. A tool is simply a function in your code that the LLM can trigger, it can pass parameters to the function, and the function should respond with some kind of output that the LLM uses to finish its response back to the user.

This is when we start getting into Agent territory, a tool on its own is still an input/output mechanism. You can most certainly run any kind of code in your function, but tools generally do not know of each other and can’t communicate with each other. So they are essentially like individual musical instruments. On their own, each tool is plenty useful, but together they have no way of sharing information or communicating.

Agents are programs that can orchestrate tools, LLMs, and even elements outside of the LLM’s scope, like creating files in a file system.

When you prompt an agent, it drafts a step-by-step plan, a sort of checklist that it must follow to achieve whatever outcome you asked it to achieve. At each of these checklist steps, the agent will ask the LLM for whatever information it needs, then proceed to call tools or other external functions until it can check that task off its checklist.

The agent will then usually verify each step, and fix any issues that occur along the way until it checks off all items in the checklist.

Agents are therefore a whole new level of automation, and you can use them to perform all kinds of tasks like building landing pages, scaffolding a coding project, web scraping, and building reports, etc.

While agents are powerful, they are still limited by the intelligence of the LLMs; thus, they still cannot think at the same level as a human and will make loads of mistakes.

I covered agents in more detail here if you want to learn more.

I have used PostgreSQL now and then throughout my career, it just feels weird to me with all the “\” commands. MySQL syntax is more English-like: “show tables; describe table …” so it’s easy to remember.

Nonetheless, PostgreSQL is a powerful open-source DB server that’s more than capable for any project size, this guide is more of a “cheat sheet” for me (and hopefully you too!).

⚠️ This article assumes you are using Linux, more specifically Ubuntu. Debian might also work, as well as other Linux server types with a few minor tweaks.

Translating MySQL commands into PostgreSQL

Connecting to a database:

\c db_name; # Postgres
use db_name; # MySQL

List databases:

\list # Postgres
show databases; # MySQL

Show tables:

SELECT tablename FROM pg_catalog.pg_tables;  # Postgres
SHOW tables; # MySQL

It’s worth noting that Postgres is going to show you all tables on the server (you can use a where clause though), whereas MySQL will just show you only tables in the current database.

Also, not to be a PostgreSQL hater, sorry but PostgreSQL will show you a colon and you need to keep pressing enter, MySQL on the other hand just lists everything.

Describe a table:

\d tablename; # Postgres
describe tablename; # MySQL

Using the CLI:

mysql -usomeuser -p1234 somedb
psql -U postgres -W somedb

With PostgreSQL, there is a user called “postgres”, so you can also run the following to access the DB with no password prompt:

sudo su - postgres
psql somedb

# OR
sudo -u postgres psql somedb

Where art thou PostgreSQL?

The configs are located in /etc/ on Linux which is very similar to MySQL. Seems there are 2 main config files to pay attention to:

Client authentication settings

This config will control who has access to your server

/etc/postgresql/16/main/pg_hba.conf

# "local" is for Unix domain socket connections only
local   all             all                                     peer
host    all             all             192.168.0.1/32          scram-sha-256
host    all             all             10.0.0.1/32             scram-sha-256

PostgreSQL works a tad bit differently than MySQL, in MySQL, you just create a user with an IP address and grant them access to whatever database you want e.g.:

# MySQL grant remote user access
create user me@192.168.0.1 indentified by '1234';

In PostgreSQL on the other hand, you must create a user without the IP and then just grant them access by listing their IP in the config file mentioned above.

⚠️ For both DB’s you still need to bind the server on 0.0.0.0 or some network IP otherwise they won’t allow external connections.

Main PostgreSQL config:

/etc/postgresql/16/main/postgresql.conf

This is the main config file where you can set the bind address, data-directory path, and various other settings such as memory settings, worker configs, row security, logging, etc…

While you can change this in the main config, the default PostgreSQL data directory is located in:

/var/lib/postgresql/

Backups

On MySQL, Percona Toolkit is my go-to toolset. You can backup, archive data and do loads more complex management tasks while the DB server is online and serving requests.

Seems like there is a version for PostgreSQL as well, I briefly read through but haven’t used PostgreSQL that comprehensively on a large DB to warrant the use of Percona.

For simple backups, you can simply run (which is similar to mysqldump):

pg_dump -U postgres -d dbname -f backup.sql

This will create a SQL dump file. I tend to avoid SQL dump files like the plague, well at least using MySQL anyway because when you restore on a new server, there could be subtle differences in the versions which can cause the restore to fail.

Furthermore, each statement has to be evaluated and run so if you have a 500GB db for example, this will be slow even if you have fast disks.

A better option is to use a filesystem backup, these are much faster and less prone to break when restoring since you copying an entire PostgreSQL instance’s directory:

pg_basebackup -D backup -Fp

• -D: the directory to store the backup in.

• -F: Format, can be plain or “tar”. Usually “p” for plain is faster but takes up more disk space. Since the “t” for tar option creates a tarball archive, this can be slow depending on how good your disk IO is.

You can then use the following to restore:

rsync -av --progress backup/ /var/lib/postgresql/16/main/
chmod 700 /var/lib/postgresql/16/main
chown -R postgres:postgres /var/lib/postgresql/16/main

Timescale

The great thing about PostgreSQL is it allows you to customize the DB via extensions, this opens up a world of possibilities, allowing the community to extend the core feature set in whatever way needed.

One such impressive extension is called Timescale, as the name suggests, Timescale allows you to store and query large time-series datasets. When querying large tables in an RDBMS (hundreds of millions of rows), you will eventually notice queries becoming incredibly slow, even with indexes and optimizations. You then would need to resort to more expensive hardware, sharding, and replication to get around these issues.

Timescale on the other hand can query and write to such tables incredibly fast, and you probably won’t even need to shard or set up replication (and be able to run on cheaper hardware too).

To install Timescale on Ubuntu for an existing PostgreSQL server, use the following script (or view the full step-by-step instructions on Timescale’s website here):

echo "deb https://packagecloud.io/timescale/timescaledb/ubuntu/ $(lsb_release -c -s) main" | sudo tee /etc/apt/sources.list.d/timescaledb.list
wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/timescaledb.gpg

sudo apt update
sudo apt install timescaledb-2-postgresql-17 postgresql-client-17
sudo timescaledb-tune


sudo systemctl restart postgresql


sudo -u postgres psql
CREATE EXTENSION IF NOT EXISTS timescaledb;

Once you have Timescale installed, you can setup a “hypertable” as follows:

CREATE TABLE meter_readings (
   time        TIMESTAMPTZ       NOT NULL,
   warehouse   TEXT              NOT NULL,
   unit_code   TEXT              NOT NULL,
   reading DOUBLE PRECISION  NULL,
);


SELECT create_hypertable('meter_readings', by_range('time'))

So basically, 95% percent is the same except for the “create_hypertable” function which sets up all the Timescale optimizations on your table.

PGVector

Another great extension is “PGVector”; if you are building a RAG-powered chatbot or need to perform similarity searches, you will need to vectorize your data and store those vector embeddings somewhere. Traditional databases like PostgreSQL and MySQL, are not designed for vector embeddings.

You will need to reach for a DB like Qdrant, which is built for that kind of storage.

Except! One problem, your project runs on PostgreSQL and it’s such a pain to introduce a new service into your stack for a dataset that’s not going to exceed maybe 10-20k worth of documents.

In such a case PGVector, while not as fast as some of these vector-optimized storage solutions, is more than sufficient for your use case.

Since it’s just a PostgreSQL extension, installing is a breeze:

git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git
cd pgvector
make
make install

Next in the PSQL shell (create extension only needs to be run once):

CREATE EXTENSION vector;

CREATE TABLE documents (
   id bigserial PRIMARY KEY,
   name VARCHAR(100) NOT NULL,
   embedding vector(1536)
);

I used “1536” because that refers to the dimensions of the vector embeddings and the size of the small text embedding model from OpenAI.

You can then insert and query data as follows:

INSERT INTO documents (embedding) VALUES ('[1,2,3...]'), ('[4,5,6....]');

SELECT * FROM items ORDER BY embedding <=> '[3,1,2...]' LIMIT 5;

The above “select” will perform a cosine similarity search and return the top 5 results. Bit of a schlep to convert those floating point numbers into the PostgreSQL format, but still far easier than maintaining another vector DB store.

Naturally, I have been test-driving Claude Sonnet for a while, and overall I must say the model is really impressive, especially for coding tasks and even creative tasks like web design.

Let’s see why I think Sonnet is one of the best, if not the best of the bigger models out in the wild right now.

Claude Sonnet is good at long-context prompts

With Sonnet, you get a decent 200k context window. This basically means you do not need to fine-tune the model. In the case of HTML templates, if you ask the model to generate you one, it’s going to be a hit-and-miss, sometimes you’ll get great-looking templates, and other times just plain junk!

To get around this you can use RAG, but for this kind of task when generating HTML templates, a full RAG solution might be overkill, instead you can just “shove” HTML as examples in your prompt similar to the below example:

<example>
...
 <nav class="navbar">
        <ul>
            <li><a href="#home">Home</a></li>
            <li><a href="#about">About</a></li>
            <li><a href="#services">Services</a></li>
            <li><a href="#contact">Contact</a></li>
        </ul>
    </nav>
...
</example>

Shoving HTML or just about any example like this will help you tame the model and get it to scope its responses to just your context.

Sonnet also supports prompt caching, therefore even though you sending this large chunk of HTML every time you prompt the model, the model does not always need to parse and analyze this every time, instead,d it’ll automatically cache this data which makes the whole process much more efficient and save you money at the same time since cached tokens are generally cheaper than regular tokens.

Furthermore, if you use their batch API instead of the real-time API, you save 50%. That’s remarkably cheap for a powerhouse model like Sonnet.

Claude Sonnet is really good at backend code

I am a bit of an old-schooler and don’t use too many AI coding assistants, I feel they generate bad code, and I love fine-grade control of the code I write.

Nonetheless, in today’s fast-paced dev world, we have customers to service. Claude Sonnet is proving to be that excellent side-kick junior developer I’ve always needed, not that I am advocating for replacing developers with AI! No Never!

Instead, AI makes developers more productive so that we can actually focus on the hardcore stuff, the engineering and system design stuff. This mundane stuff that took up time before, like fixing some CSS issue, frankly for a backend dev is a waste of time.

Even in the engineering process, there are often times you would “float” (for lack of a better word) between concepts, which means now you have to scaffold 2-3 variations which could take a couple of hours or days.

With Sonnet, you can just prompt the model to scaffold pseudo-code versions and then iterate from there. It’s boilerplates on steroids essentially!

Previously with ChatGPT, you could get the model to do something similar, however, often it was a waste of time.

The code generated was either heavily hallucinated or outdated, you just couldn’t get a consistent generation from those models and wasted too much time fine-tuning the prompt.

Sonnet seems to have a decent level of accuracy, it’s not always perfect but often it gets you there much closer than any of the OpenAI models ever could.

Claude Sonnet is not bad at web design

I can do UI development just fine, I know my way around pure CSS, Bootstrap, and even Tailwind, yet building frontends is not really my cup of tea, I am backend’er and prefer it that way.

Still, from time to time I have no choice but to throw together a simple landing page or tweak some components here and there.

Previously, well I had two options write the code myself or just buy a template. Now with Sonnet, and a really well-structured prompt, I can get at least 80% there in just a few minutes, and then fine-tune the design manually as and where needed.

A complete landing page from a simple prompt

It’s amazing how much time you could save with Sonnet; I have done a few experiments with generating layouts and UI components and for the most part, Sonnet does a pretty good job.

When the generated code isn’t up to standard, you can step-by-step ask Sonnet to tweak certain aspects, similar to sitting with a web designer and giving them feedback.

Here’s an example of a full layout designed 100% by Sonnet:

Not bad for a business profile type website right? Sure there are some rough edges, and some styling fine-tuning that one can do to make it a little bit more “pretty” but hey this was generated in less than a minute, and it’ll take me 10 minutes just to go in and tweak versus waiting for a designer or scrolling through marketplaces looking for a paid template.

The prompt (I totally made up the plumbing company name, although Sonnet seems to have stolen real content from a real plumbing website, https://danvilleplumbers.com/ in this case, not sponsored but please support them if you are in the area 😊):

Design me a modern looking landing page to 
market my plumbing services. You must have an 
About me section, my services section, 
contact me section including a nice big looking footter. 
My colors are: rgb(220, 38, 38), black and white. 
Use a parallax effect and images from unsplash.com. 
Generate the content for each section as well. 
My company name is Danville Plumbers. 
Make sure you have a lot of white space and use pictures 
that are relevant to plumbing. 
Only return HTML, and no commentary or extra text 
so that it's easy for me to copy and paste.

And the generated HTML:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Danville Plumbers - Professional Plumbing Services</title>
    <style>
        * {
            margin: 0;
            padding: 0;
            box-sizing: border-box;
            font-family: 'Arial', sans-serif;
        }

        html {
            scroll-behavior: smooth;
        }

        .navbar {
            position: fixed;
            width: 100%;
            background-color: rgba(0, 0, 0, 0.9);
            padding: 20px;
            z-index: 1000;
        }

        .navbar ul {
            list-style: none;
            display: flex;
            justify-content: flex-end;
            margin-right: 50px;
        }

        .navbar li {
            margin: 0 20px;
        }

        .navbar a {
            color: white;
            text-decoration: none;
            font-weight: bold;
            transition: color 0.3s;
        }

        .navbar a:hover {
            color: rgb(220, 38, 38);
        }

        .hero {
            height: 100vh;
            background: linear-gradient(rgba(0, 0, 0, 0.7), rgba(0, 0, 0, 0.7)),
                        url('https://images.unsplash.com/photo-1584622650111-993a426fbf0a?ixlib=rb-4.0.3');
            background-attachment: fixed;
            background-size: cover;
            display: flex;
            align-items: center;
            justify-content: center;
            color: white;
            text-align: center;
        }

        .hero-content h1 {
            font-size: 4em;
            margin-bottom: 20px;
        }

        .hero-content p {
            font-size: 1.5em;
            margin-bottom: 30px;
        }

        .cta-button {
            padding: 15px 30px;
            background-color: rgb(220, 38, 38);
            color: white;
            text-decoration: none;
            border-radius: 5px;
            font-weight: bold;
            transition: background-color 0.3s;
        }

        .cta-button:hover {
            background-color: rgb(185, 28, 28);
        }

        section {
            padding: 100px 20%;
        }

        .about {
            background-color: white;
        }

        .services {
            background-color: #f5f5f5;
        }

        .service-grid {
            display: grid;
            grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
            gap: 30px;
            margin-top: 50px;
        }

        .service-card {
            background-color: white;
            padding: 30px;
            border-radius: 10px;
            box-shadow: 0 5px 15px rgba(0,0,0,0.1);
            transition: transform 0.3s;
        }

        .service-card:hover {
            transform: translateY(-10px);
        }

        .contact {
            background: linear-gradient(rgba(0, 0, 0, 0.8), rgba(0, 0, 0, 0.8)),
                        url('https://images.unsplash.com/photo-1599567832218-907816219e67?ixlib=rb-4.0.3');
            background-attachment: fixed;
            background-size: cover;
            color: white;
        }

        .contact-form {
            display: grid;
            gap: 20px;
            max-width: 600px;
            margin: 0 auto;
        }

        input, textarea {
            padding: 15px;
            border: none;
            border-radius: 5px;
        }

        footer {
            background-color: black;
            color: white;
            padding: 50px 20%;
            text-align: center;
        }

        .footer-content {
            display: grid;
            grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
            gap: 30px;
            margin-bottom: 30px;
        }

        h2 {
            margin-bottom: 30px;
            font-size: 2.5em;
        }

        p {
            line-height: 1.6;
            margin-bottom: 20px;
        }
    </style>
</head>
<body>
    <nav class="navbar">
        <ul>
            <li><a href="#home">Home</a></li>
            <li><a href="#about">About</a></li>
            <li><a href="#services">Services</a></li>
            <li><a href="#contact">Contact</a></li>
        </ul>
    </nav>

    <div class="hero" id="home">
        <div class="hero-content">
            <h1>Danville Plumbers</h1>
            <p>Your Trusted Plumbing Experts Since 1995</p>
            <a href="#contact" class="cta-button">Get Free Quote</a>
        </div>
    </div>

    <section class="about" id="about">
        <h2>About Us</h2>
        <p>With over 25 years of experience, Danville Plumbers has been the go-to plumbing service provider in the Danville area. We pride ourselves on delivering exceptional quality work with unmatched customer service.</p>
        <p>Our team of licensed professionals is available 24/7 for emergency services, ensuring that your plumbing needs are met whenever they arise. We believe in transparent pricing, honest service, and building long-lasting relationships with our clients.</p>
    </section>

    <section class="services" id="services">
        <h2>Our Services</h2>
        <div class="service-grid">
            <div class="service-card">
                <h3>Emergency Repairs</h3>
                <p>24/7 emergency plumbing services for any urgent situations.</p>
            </div>
            <div class="service-card">
                <h3>Installation</h3>
                <p>Professional installation of fixtures, pipes, and plumbing systems.</p>
            </div>
            <div class="service-card">
                <h3>Maintenance</h3>
                <p>Regular maintenance and inspection services to prevent future issues.</p>
            </div>
        </div>
    </section>

    <section class="contact" id="contact">
        <h2>Contact Us</h2>
        <form class="contact-form">
            <input type="text" placeholder="Your Name" required>
            <input type="email" placeholder="Your Email" required>
            <input type="tel" placeholder="Your Phone">
            <textarea rows="5" placeholder="Your Message" required></textarea>
            <button type="submit" class="cta-button">Send Message</button>
        </form>
    </section>

    <footer>
        <div class="footer-content">
            <div>
                <h3>Danville Plumbers</h3>
                <p>123 Main Street<br>Somwhere, Someville</p>
            </div>
            <div>
                <h3>Hours</h3>
                <p>Monday-Friday: 8am-6pm<br>
                   Emergency Service: 24/7</p>
            </div>
            <div>
                <h3>Contact</h3>
                <p>Phone: (965) 777-0123<br>
                   Email: info@xyz.com</p>
            </div>
        </div>
        <p>&copy; 2023 Danville Plumbers. All rights reserved.</p>
    </footer>
</body>
</html>

Not just static websites

With clever prompting, you can even get Sonnet to code in any backend language, including writing the email code for sending forms or building a gallery.

I am not a Figma person (nor a frontend person for that matter), I do not usually draw layouts and concepts first unless it’s a large project, thus usually, I’ll just take an hour or two to “throw together” a POC and then fiddle with that until I get something closer to the finished concept.

Now with Sonnet, within a few minutes, I can scaffold designs, get variations, and play around with different concepts in a fraction of the time. It’s almost like the Ruby-on-Rails scaffold era, where you could spin up a whole blog in minutes.

Don’t worry though; AI is not going to replace designers or developers, it cannot really think or create at the same level as we do, but it can give creatives (or technical) people a powerful paintbrush to scaffold ideas really quickly.

ChatGPT was okayish at one point, but the quality of the code seemed to drop over time and half the time that code wouldn’t work, Sonnet on the other is far better at generating good quality code that actually works!

Claude Sonnet’s not perfect: Anthropic has scaling issues

I have always noticed weird scaling issues with Anthropic (model overloaded) and even times when the API returned blank results. I find OpenAI is much more stable and GPT4o-mini is much cheaper as well.

Thus, for more RAG-type applications or high-traffic use cases, I tend to use OpenAI models. For creative and as a coding assistant, Sonnet stability isn’t much of an issue.

Another annoying thing with Anthropic is that you have to follow the conversation sequence: AI-HUMAN-AI-HUMAN, this is not natural in the real world, especially when dealing with Voice AI.

OpenAI just seems more idiot proof 😂.

Hopefully Anthropic can work on the developer experience, I feel that’s really holding them back because their models are really great for creative tasks, usually better than OpenAI models.

Vercel is a great platform, it works and is convenient. Most developers hate dealing with servers and are sometimes scared of touching Linux, so it makes sense to outsource this aspect to DevOps specialists and just focus on coding.

This approach works well initially, but as you grow, costs can escalate, sometimes exponentially! So why not just roll out your own servers? It might be a few hours of work SSH hardening and setting up, but once it’s running, these servers can go for years without the need for any additional maintenance.

If you want to go down the self-hosted route but don’t know where to start, then this article is perfect for you. I will go through it step-by-step.

Note: This tutorial assumes you using Ubuntu Server for your instances. Debian might work as well.

Understanding the costs

Before we even think about setting up a server, let’s talk about the costs. When you host with PaaS providers like Vercel, they usually charge you for storage and compute separately.

You, therefore, will pay for how much memory your application consumes, and how long it runs for, plus any storage you use. For a low-traffic site, this can be really affordable because when you have little to no requests, your costs are relatively negligible.

A VPS/dedicated server on the other hand tends to come with a fixed cost, I use Hetzner, thus I will base my figures on their current pricing but should be similar to other hosting companies such as Digital Ocean or AWS.

On Hetzner, you can get decently sized VPS servers for (Not sponsored, I use them personally for my sites):

With a VPS instance, you get everything you need including network traffic, disk space, and usually better RAM and CPU allocations. In addition, you have full control of your instance and can run anything you desire.

The only caveat is that VPS providers do limit your traffic allocation per month, so if you exceed 20TB in one month (in the Hetzner case above), you’ll pay per GIG for the extra usage.

Realistically though, if you are getting more than 20 TBs of traffic, these VPS servers are still going to be way cheaper than something like Vercel.

Let’s learn some basic Nano

I am sorry, that this may seem oddly misplaced in the context of this article, but I wanted to get it out the gate as quickly as possible. Linux uses config files for nearly every application, so a basic understanding of terminal-based text editors is essential. If you are already familiar with Nano, Vi, VIM, etc.… you can safely skip this section.

I can’t remember if it comes pre-installed with Ubuntu, but if not simply run:

apt update
apt install nano

Nano commands:

• nano file_name - This will open the specified file path in the editor.

• Arrow keys to move up or down, or left or right in the editor.

• “ctrl+o” to save any changes.

• “ctrl+x” to exit.

• “ctrl+/” go to any line number e.g. 10

• “ctrl+c” cancels an operation.

• “ctrl+w” search through the file.

• if you use: “nano +5 file_name”, it’ll open the file and move the cursor to the 5th line.

There you go, simple, right? At a bare minimum, you only need to know “ctrl+o” and “ctrl+x” to edit, save, and exit.

Initial setup

The first thing you want to do is to generate SSH keys, this is a much more secure way of accessing your server compared to regular passwords. On most terminals, you can run:

ℹ️ SSH Keys are a pair of long encrypted "secret codes" i.e. a private key and public key. These are much more secure and harder to crack compared to regular old passwords. The server will keep a copy of your public key, and match that against the private key you send when trying to SSH into the server, if there are any discrepancies, that connection will be automatically denied.

ssh-keygen -t rsa -b 4096

The above command will generate a private and public key pair. You will need to copy the “.pub” file’s contents and paste it into the server’s “~/.ssh/authorized_keys” file, most providers offer a simple GUI to do this.

ℹ️ ~/ is just a shortcut for /home/username or the home folder for the currently logged in user on the Linux terminal.

On Hetzner, you can upload this key under this section when creating a server (I blurbed out my key names for privacy):

Finally, once the server has been created, you will get a public IP. To access the machine, just SSH in:

ssh root@192.x.y.z

This should automatically pick up the key you created and log you in, if you do encounter issues, try the following:

ssh -o IdentitiesOnly=yes -i /path/to/your/private/key root@192.x.y.z

You may also see the error below with a new server, the warning is a bit “dramatic” to say the least. You should verify that you are using the correct IP and the key is correct but usually, this is just a sanity check by the SSH daemon: “I am seeing this server IP for the first time, and I don’t know if I can trust it, so I will refuse and print a scary message!?”

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:g8DZMXxNy8a7fM4/Xyz.

To fix this, simply run the following to trust this new server:

ssh-keygen -f "/home/$USER/.ssh/known_hosts" -R "x.y.z"

Securing the server

When it comes to VPS servers you should use the following security best practices:

• Use both a network and server firewall. Most providers including Hetzner have a concept of a network firewall (also known as a cloud firewall or WAF). You should set one up as soon as you provision your instance. Open ports (TCP): 22, 443, 80 and block everything else.

• Change the default SSH port. Hackers can sniff the new port, but most attacks come from automated scripts that target port 22, by simply changing this port, you are protecting yourself from loads of automated tools.

• Disable root access and create an SSH-only user.

• Use a VPN, this is optional but highly recommended. You can then open ports only to your VPN keeping your server ports invisible to the rest of the internet.

Next, let’s go through each of the mentioned security steps.

Setting up a network firewall

This will differ from provider to provider; I will just give you an example using Hetzner. In the Hetzner cloud console, click on “Firewalls” in the left navigation (Cloud firewalls are free to use).

You should see the following screen. Simply just allow TCP 22, 80, and 443. In my case, I also added “9222”. This is because, in the next few steps, we’ll change the default SSH port to “9222”. You can choose any random port you want, just make sure it doesn’t clash with any process running on your server.

Whenever you create an instance in the future, you should see the “firewalls” option, just click on the one you previously created to attach the instance to that firewall:

Alternatively, for existing servers. Just click on the “Firewalls” tab in the instance’s details page and select “Apply Firewall” to add the server to one of your existing firewalls:

Setting up a server firewall

It’s also a good practice to set up a firewall on your VPS instance as well, it’s highly unlikely that network traffic will bypass the main firewall, but just in case, you can never be too sure!

First, you will need to SSH into the server:

ssh root@your_ip

UFW is the default firewall on Ubuntu when you login into the remote server, you can confirm that UFW is installed by typing:

ufw

You should get the “help” menu, if not just run:

apt update -y
apt install ufw -y

Let’s allow the required ports as follows:

ufw allow http
ufw allow https
ufw allow ssh
ufw allow from any to any port 9222
ufw enable

You’ll be asked to confirm, just hit “y”. Now if you run:

ufw status numbered
###### You should see #######

Status: active

To                         Action      From
--                         ------      ----
[ 1] 80/tcp                     ALLOW IN    Anywhere                  
[ 2] 22/tcp                     ALLOW IN    Anywhere                  
[ 3] 9222                       ALLOW IN    Anywhere                  
[ 4] 80/tcp (v6)                ALLOW IN    Anywhere (v6)             
[ 5] 22/tcp (v6)                ALLOW IN    Anywhere (v6)             
[ 6] 9222 (v6)                  ALLOW IN    Anywhere (v6)

Awesome! We now have a decently secured server. If you opted for the VPN option, you can do the following instead:

ufw allow http
ufw allow https
ufw allow from x.x.x.x to any port 22
ufw allow from x.x.x.x to any port 9222
ufw enable

Naturally “x.x.x.x” should be replaced by your VPN’s IP address.

Note: I use “ufw status numbered” because this also prints the “index” of the rule, in case we need to delete that rule. You can totally omit “numbered” and this command will work fine.

Changing the default SSH port

The network firewall can be altered at any time, however, the instance firewall cannot, therefore, it’s advisable to keep one tab open with an active SSH connection and also that you allow both 22 and 9222 [or whatever port you choose] first, then test the new port works, and thereafter drop the old 22 from your firewall.

To change the port do the following (or you can just nano /etc/ssh/sshd_config and manually change it):

# Backup original file just incase
cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup

# Set the new port
sed -i 's/\#Port 22/Port 9222/' /etc/ssh/sshd_config

# Restart ssh
systemctl restart ssh

Add another layer of security

Brute force attacks are also another common security issue when you host a publically accessible server, by brute force, I mean a bot that will constantly try hundreds or thousands of different passwords or key combinations to try and break into your server.

A good layer of protection for this kind of attack is to use Fail2Ban. Fail2Ban will basically watch common ports like 22 for abuse traffic and block them automatically.

To setup Fail2Ban:

sudo apt install fail2ban
# So we can customize fail2ban settings
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

sudo nano /etc/fail2ban/jail.local

Next, paste it into the config file:

[DEFAULT]
bantime = 1h
findtime = 10m
maxretry = 5
banaction = iptables-multiport

ignoreip = 127.0.0.1/8 ::1
destemail = your@email.com

[sshd]
enabled = true
port = 9222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 1h

And finally, restart:

sudo systemctl restart fail2ban

You can do much more advanced stuff like preventing DDOS attacks on port 443, 80, etc…, for no,w though we are basically just protecting port 9222. If abuse is detected, Fail2Ban will block that IP for 1 hour.

Setting up an SSH-only user

SSH’ing as root is a bad idea, If this user is compromised then the attacker can access your whole system, instead, you should create a password-based user that can elevate their privileges to sudo only when needed.

adduser yourusername
usermod -aG sudo secureuser

Next, you should copy your ssh pub key to ~/.ssh/authorized_keys:

sudo su - secureuser

mkdir ~/.ssh
# Paste your pub key with
nano ~/.ssh/authorized_keys

# Fix permissions
chmod 600 ~/.ssh/authorized_keys
chmod 700 ~/.ssh

Now you should in a separate terminal, try to ssh as this user:

ssh -p 9222 secureuser@server_ip

🤞The above should allow you in, alternatively check that there are no spaces in the authorized file and retry setting the permissions, usually it’s either bad permissions or the public key is invalid.

You can also use “-i” with your SSH command to specify which key you want to use:

ssh -i ~/.ssh/id_rsa -p 9222 secureuser@server_ip

Almost there, we want to do two more security optimizations:

• Prevent the root user from logging in.

• Disable password logins. Earlier we created the SSH user with a password, this password is not meant for SSH access, instead, it’s just there to add an extra layer of security. Should an attacker gain access to this SSH user, it still makes it a little more difficult to access root, since running “sudo su - “ will prompt for the password.

To action:

nano /etc/ssh/sshd_config
# Uncomment and change these lines to:
PasswordAuthentication no
PermitRootLogin no

Finally, restart the ssh daemon:

systemctl restart ssh

Setting up docker

Great! Now you have a fairly secure server up and running, but to run Next.js or your application code, you probably gonna need to install docker first. Installing Docker is a breeze:

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

Now when you type the following, you should see docker installed:

docker ps

It is always a good idea to run docker as a none root user, you can do so as follows:

# You can call the user whatever you want
sudo useradd -m -s /bin/bash dockeruser
sudo usermod -aG docker dockeruser

Now, you should log in with this user and run your docker containers as “dockeruser”:

sudo su - dockeruser

Conclusion

There you go, a detailed step-by-step guide, copy-and-paste basically of all the steps you need to run your own VPS server, it may seem like a lot of steps but it’s like 99% the same for every server you setup, so it becomes muscle memory after a while.

What if I could build an AI voice phone just for fun? Something that she could interact with. This would be way cooler than those stuffed animal toys that keep repeating the same sentences over and over. Since this “toy” is powered by a full LLM, it could actually have an entire conversation and role-play as if it were an actual cartoon character!

💡 Are you new to Python? Learn Python with LearnPython.com courses. Fun interactive courses that are based on real-life business scenarios, meaning you’ll be writing Python code and seeing the results instantly. No need to install Python or other tools on your device, everything happens through your favorite web browser (Sponsored content).

In this article, I will walk you through step-by-step how I achieved this and all the relevant technologies and services needed.

WebRTC VoiceAI: How will this work?

WebRTC is a browser peer-to-peer standard that allows just about any modern browser to stream rich media content such as video or audio in real-time, similar to a voice or video call.

Thus, WebRTC is perfect for our little Voice AI character (Aka ZaZu the Monkey!). For the backend, we will use a WebSocket server built in Python, which orchestrates the AI chain to transcribe the audio, prompt the model, and finally convert the model’s response back to audio and stream it back to the browser using WebRTC.

Now, since this is a fun app, I wasn’t going to spend a lot of time building my own transcription model, or TTS model for that matter. Instead, we’ll just use Assembly AI for transcription and OpenAI for everything else.

I initially tried to build this with VOSK instead of Assembly AI which is a decent open-source speech-to-text library, however, the transcription quality wasn’t that great, and it wasn’t worth the time and effort to train a custom VOSK model. AssemblyAI has really good accuracy and costs just $0.37 per hour, which is insanely cheap!

You can view the full source code here: https://github.com/kevincoder-co-za/zazu-voiceai

⚠️ I omitted “try except” blocks throughout to make the code more readable, however, for a real production app, you should catch and handle errors more efficiently.

Setting up our WebSocket server

For the WebSocket server, we’ll go a bit old-school and use Flask. Flask has very good support for web sockets and is fairly easy to spin up a quick prototype. To get started, let’s install a few pip packages:

pip install -r requirements.txt

flask
flask-sockets
gevent
gevent-websocket
assemblyai
ffmpeg

You will in addition need to install “FFMpeg” for your operating system. FFmpeg is a powerful audio manipulation library used by most modern audio tools on the market, and we’ll use it to alter the audio generated by OpenAI to sound more cartoonish.

Next, you’ll want to create a Python file: “server.py” or whatever you prefer:

from flask import Flask, render_template
from flask_sockets import Sockets
from gevent import pywsgi
from geventwebsocket.handler import WebSocketHandler
from audio_handlers import AudioHandler
from llm_adapters import OpenAILLM
import settings

app = Flask(__name__)
sockets = Sockets(app)

@app.route("/dialer")
def dialer():
    return render_template("dialer.html", 
        sample_rate = settings.SAMPLE_RATE,
        socket_url = settings.SOCKET_URL
    )

@sockets.route("/websocket/stream")
def audio_stream(ws):
    audio_handler = None
    llm = None

    while not ws.closed:
        message = ws.receive()

        if message is None:
            continue
        if isinstance(message, str):
            if "start call" in message:
                print("Call started", flush=True)
                llm = OpenAILLM()
                audio_handler = AudioHandler(llm, ws)
            elif "end call" in message and audio_handler:
                audio_handler.stop()
                llm = None

        elif isinstance(message, bytes) or isinstance(message, bytearray):
            audio_handler.stream(bytes(message))

if __name__ == "__main__":
    WEBSOCKET_PORT = 5000
    server = pywsgi.WSGIServer(("", WEBSOCKET_PORT), app, handler_class=WebSocketHandler)
    print(f"Server listening on ws://0.0.0.0:{WEBSOCKET_PORT}")
    server.serve_forever()

Let’s go through this step-by-step, first, we set up our socket server and two routes:

app = Flask(__name__)
sockets = Sockets(app)

@app.route("/dialer")
def dialer():
    return render_template("dialer.html", 
        sample_rate = settings.SAMPLE_RATE,
        socket_url = settings.SOCKET_URL
    )

@sockets.route("/websocket/stream")
def audio_stream(ws):

The dialer route just returns an HTML page: “dialer.html”. This contains our JavaScript WebRTC phone implementation. We’ll get to that in a bit.

The “/websocket/stream” is where all the magic happens. When the browser initiates a call, this endpoint is triggered, and a socket is opened. Since the socket will remain open until either party terminates the call, we can therefore constantly stream data back and forth in a consistent manner.

@sockets.route("/websocket/stream")
def audio_stream(ws):
    audio_handler = None
    llm = None

    while not ws.closed:
        message = ws.receive()

        if message is None:
            continue
        if isinstance(message, str):
            if "start call" in message:
                print("Call started", flush=True)
                llm = OpenAILLM()
                audio_handler = AudioHandler(llm, ws)
            elif "end call" in message and audio_handler:
                audio_handler.stop()
                llm = None

        elif isinstance(message, bytes) or isinstance(message, bytearray):
            audio_handler.stream(bytes(message))

In our “audio_stream” method, we scan the incoming data to check if it’s raw bytes or a string. When the data is raw bytes, this means that we are receiving audio data and when the data is a string, this means we are receiving an event such as “start call” or “end call”.

When the call starts, we set up an audio_handler and llm. The “audio_handler” will transcribe audio and send it back to the user.

The “llm” will communicate with GPT4o-mini and generate an appropriate response to the user based on the transcription.

Building the LLM Adapter

The LLM adapter has two tasks it needs to perform:

• Prompt the text model with the transcription and get back a generated response (invoke method).

• Take the generated response and convert it to audio so that we can stream it back to the browser (text_to_audio method).

from openai import OpenAI
import settings

class OpenAILLM:
    def __init__(self):
        self.llm = OpenAI()
        self.messages = [
            ("system",settings.SYSTEM_AI_PROMPT)
        ]

    def invoke(self, message):
        self.messages.append(("user", message))
        messages = self.messages.copy()

        chat_completion = self.llm.chat.completions.create(
            model= settings.OPENAI_GPT_MODEL,
            messages=[{"role": m[0], "content": m[1]} for m in messages]
        )

        response = chat_completion.choices[0].message.content
        self.messages.append(("assistant", response))

        return response

    def text_to_audio(self, text, audio_file_path):
        response = self.llm.audio.speech.create(
            model=settings.OPENAI_TTS_MODEL,
            voice=settings.OPENAI_TTS_VOICE,
            input=text
        )

        response.stream_to_file(audio_file_path)

Note: We also have some memory built in so that the LLM has context for the entire conversation. To maintain memory, we simply just append all messages to a list of “messages” and then use the conversation API to send a batch of messages.

Building the Audio Manager

The Audio Manager can be found in “audio_handlers.py”:

import os
import uuid
import settings
import assemblyai as aai
import ffmpeg

aai.settings.api_key = settings.ASSEMBLYAI_API_KEY

class AudioHandler(aai.RealtimeTranscriber):
    def __init__(self, llm, ws):
        super().__init__(
            on_data=self.on_data,
            sample_rate=settings.SAMPLE_RATE,
            on_error=lambda x : print(x, flush=True),
            disable_partial_transcripts=True
        )
        self.llm = llm
        self.ws = ws
        self.connect()

    def stop(self):
        self.close()

    def on_data(self, transcript: aai.RealtimeTranscript):
       if isinstance(transcript, aai.RealtimeFinalTranscript) and transcript.text:
            response = self.llm.invoke(transcript.text)
            if response != "":
                self.respond_to_user_prompt(response)

    def respond_to_user_prompt(self, transcript):
        tmp_id = str(uuid.uuid4())
        tmp_file = f"/tmp/_{tmp_id}.mp3"
        cartoonified_version = f"/tmp/modified_{tmp_id}.mp3"

        self.llm.text_to_audio(transcript, tmp_file)        

        ffmpeg.input(tmp_file).filter('asetrate', 33050).output(cartoonified_version).run()

        audio_data = None

        with open(cartoonified_version, "rb") as f:
            audio_data = f.read()

        if os.path.exists(tmp_file):
            os.remove(tmp_file)
        if os.path.exists(cartoonified_version):
            os.remove(cartoonified_version)

        if audio_data:
            self.ws.send(audio_data)

The audio manager class extends Assembly AI’s RealtimeTranscriber. We override the “on_data” method to grab the finished transcription and then use both OpenAI Whisper and FFMpeg to generate a cartoonish voice (You probably should use a voice cloning API for better performance).

    def __init__(self, llm, ws):
        super().__init__(
            on_data=self.on_data,
            sample_rate=settings.SAMPLE_RATE,
            on_error=lambda x : print(x, flush=True),
            disable_partial_transcripts=True
        )
        self.llm = llm
        self.ws = ws
        self.connect()

In the constructor, we set up a few configs:

• on_data: This method will allow us to receive the full transcription and perform any other post-processing needed.

• sample_rate: Measured in hertz and basically the number of audio samples carried per second. Assembly AI supports a wide variety of sample rates, but we want to keep a consistent sample rate between the different services (OpenAI Whisper only supports 16 kHz, hence our sample rate is 16000).

• disable_partial_transcripts: for this application, we don’t need phrases or parts of speech; all we care about is the full transcript. Turning this setting on will minimize the network requests between us and Assembly AI, thus improving performance.

Once we get a transcription back from Assembly AI, we then need to:

• Prompt the LLM to generate a response.

• Send the response to the OpenAI TTS model.

• Receive the audio from OpenAI TTS and then use FFMpeg to alter the audio so that it sounds similar to a cartoonish voice. OpenAI does not support custom voices, so we are stuck with human voices and therefore need to alter the pitch to sound similar to a cartoon character.

Silent pauses are a pain!

One of the major pain points is the “silent pause” which occurs for a few milliseconds (or seconds in some instances) after the user stops speaking, this is the time taken for the audio transcriber to detect the end of a sentence and also all the AI queries back and forth to generate the final response back to the caller.

This can be annoying since it’s not a fluid conversation that you would typically have with a real person.

Nonetheless, it works pretty decently if you can stomach the annoying pauses.

A few approaches I will look at to improve the code when I get some more time:

• Rebuild in a faster language, I actually already done this partially in Golang. It does make a difference, but it could still do with more optimizations.

• Run my own model near the WebSocket server. PHI-3 is fairly good and fast, running this model with Ollama and then fine-tuning could really improve both the speed and accuracy of the AI queries.

• Train my own voice model, similar to the above. It might be better to just fine-tune a speech model from scratch. I’ve played around with pocketsphinx and Whisper, both are great options and not complicated to train.

• Implement using a speech-to-speech model. These improve response times drastically, but can be quite pricy at this stage.

Conclusion

While this “toy app” is not much at this stage, it’s useful enough to amuse my kid 😀 and a great base for me to build upon a more commercial offering later on. I see Voice AI growing in the next few years and being very useful in environments such as call centers or for building custom business context-aware agents.

The great news is that the technology gets better every few months, and we are spoiled for choice. At the time of writing, I noticed several new speech-to-speech API’s springing up, including OpenAI’s real-time models. The only caveat is that they are just too expensive to run. Elevenlabs is probably the most cost-effective option at this stage at around $99 per 1000 minutes of audio.

I plan to give Elevenlabs a try, and also Deepgram agents will be interesting as well.

Python is cool; it's compact, versatile, and has a broad spectrum of use cases, especially in web development and machine learning. Django is well-designed with batteries included and is one of the most mature frameworks around.

Since I have worked extensively with both frameworks and languages, this gives me a unique perspective on things. In this article, I will give you a general overview of both frameworks to help you choose the right one for your needs.

Which is better?

Neither, there is no better stack. Both frameworks are an excellent choice for most use cases.

Disappointing? Sorry, I know you came here to get a final verdict on which is better, but, my goal is to give you a deep technical review of features and functionality so that you can make an informed decision and not base your decision solely on hype or opinions.

Destroy the world!?

Before we continue with more framework-related stuff, it's important to understand one of the most fundamental differences between PHP and Python.

PHP spawns up a worker process that loads Laravel "from scratch" on every request, thus the lifecycle for a Laravel app begins when a request is made and ends when the worker closes the request.

The PHP engine does use a JIT compiler and some fancy caching optimizations to make this process fast, so it's not exactly "from scratch" each time, however, if you declare a global variable, this variable gets destroyed when the request is finished.

Python on the other hand loads Django into memory once at startup, thus any global variables that exist on startup will persist throughout every request until the application restarts.

This is why when you change a PHP file, it immediately updates on the next request without the need for a restart. In Django, you will most likely need to restart the Gunicorn workers for the new code to take effect.

This is great for performance since Django has everything it needs already loaded, Django can thus serve up requests much faster. Not so great if you want to run your application in a shared environment though, such as Cpanel hosting, since each Django instance is bootstrapped once, the entire application will sit in memory all the time waiting for requests.

The PHP application on the other hand only uses up resources (besides a small percentage for caching) when there are requests.

PHP does have alternative runtimes like octane and frankenphp, which work similar to Django's load once in memory approach, however, PHP-FPM is the default and most commonly used runtime in the PHP world.

Package management and setting up

Package management is a vital component in modern development, you almost always will need to pull down at least one external package in your project at some point in time. Both PHP and Python have excellent package managers.

Python’s most popular package manager is PIP. To use PIP you first need a virtual environment:

python3 -m venv venv
source venv/bin/activate

The above commands set up a virtual environment that isolates your project dependencies from the system's Python (a very good idea on Linux! Since the OS heavily relies on Python). Furthermore, you can keep a separate virtual environment for each project.

Now you can run the following to install Django and set up a Django project :

pip instal django
django-admin startproject myproject

PHP has a similar setup process, except PHP uses Composer as its package manager which you can download and install from here. You will need to have PHP installed prior. If you are on Mac or Windows, you basically can skip all of these steps and just download herd. Herd is a Laravel-provided toolset that installs everything you need to build a Laravel application, including the relevant DB servers.

Once composer is installed on your system, you can install Laravel as follows:

composer create-project laravel/laravel myprojectname --prefer-dist

Composer will automatically configure itself within the "myprojectname" project and create a "vendor" folder which is similar to the virtual environment setup in Python.

The only difference is that the "vendor" folder just contains PHP packages, not an entire PHP runtime.

Python copies the whole Python runtime to the virtual environment, thus it's completely isolated (except for C libraries installed via your package manager).

To install a package via composer:

composer require vendor/package

//e.g.
composer require guzzle/guzzle

Eloquent

Laravel has a great ORM called eloquent; it's very close to raw SQL and feels natural for someone like me, who prefers writing SQL. I like using native DB functionality and optimizing queries by hand so I enjoy writing queries using Eloquent.

Here is an example model:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;

class Product extends Model
{

}

Eloquent will automatically associate this model with a " products" table in our database. Furthermore, you do not have to explicitly declare every table field because when you create an instance of this class: the fields automatically become available on instances of the class e.g.: "$product->price".

To start querying the model you can then try the following:

Product::where("price", ">=", 100)
    ->skip(5)
    ->take(10)
    ->get()

This is an elementary example but should give you an idea of how easy it is to use Eloquent.

The great thing about Laravel, in general, is how flexible the framework is, for example, if you do not want to declare a model class, you could actually just use the database facade directly:

use Illuminate\Support\Facades\DB;
DB::table("products")->where("price", ">=", 100)
    ->skip(5)
    ->take(10)
    ->get()

You can also just use regular old RAW SQL as follows:

$products = DB::select("select statement here")

Django ORM

Django has a similar concept of models, however, you have to explicitly declare each field:

import os
from django.db import models

class Product(models.Model):
    name = models.CharField(max_length=255)
    price = DecimalField(max_digits = 5, decimal_places = 2) 
    visible = models.BooleanField(default=False)

    def __str__(self):
        return self.name

Similar to Eloquent, the Django ORM will automatically look for a table named "products".

I constantly forget the fields, especially on larger tables, thus I kinda like this way of doing things since my editor will now have table field definitions to provide IntelliSense suggestions.

To query the Django ORM is a bit weird compared to regular SQL, but because Python is so easy to read, it's not that bad at all.

Product.objects.filter(price__gte=100)[5:15]

To perform raw queries you can use the ".raw" instead of "filter":

query = """SELECT * FROM myapp_product WHERE price >= %s 
ORDER BY price LIMIT 10 OFFSET 5"""
products = Product.objects.raw(query, [100])

You can also use the connection cursor directly for more advanced queries:

from django.db import connection

def get_obj_using_raw_sql(sql, binds):
    with connection.cursor() as cursor:
        cursor.execute("sql here", binds)
        row = cursor.fetchone()

    return row

Migrations

Both frameworks provide great tooling for migrations, in Laravel you have to explicitly write out your migration files:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

class CreateUsersTable extends Migration
{
    public function up()
    {
        Schema::create('users', function (Blueprint $table) {
            $table->id();
            $table->string('name');
            $table->string('email')->unique();
            $table->timestamp('email_verified_at')->nullable();
            $table->string('password');
            $table->rememberToken();
            $table->timestamps();
        });
    }
    public function down()
    {
        Schema::dropIfExists('users');
    }
}

Next, run the following command to apply these migrations against the database:

php artisan migrate

In Django, since you already declared all fields in the "models.py" file, you do not need to explicitly write out a migration. Simply run:

python manage.py makemigrations
python migrate

Every time you update a model, simply repeat the above and Django will automatically update your DB to be in sync with the model.

The verdict on models & migrations

I like Django's model and migration structure, it is such a fluid experience just to declare fields and run two commands versus having to manually write out a migrations file every time.

With Django's ORM, as queries get more advanced, you will need to start using "Q", "annotate", "aggregate" and so forth. It becomes a little cumbersome to work with compared to Eloquent which keeps that more natural "SQL" like feel regardless of how complicated the query gets. Thus, for querying, I prefer Eloquent.

While this is not exactly advanced, it's sufficient to give you an idea:

Product::where("price", ">=", 200)
    ->orWhere("discount", ">=", "20")
    ->sum("price");

In Django:

from myapp.models import Product
from django.db.models import Q
from django.db.models import Sum

products = Product.objects.filter(
     Q(price__gte=200) | Q(discount__gte=20)
).aggregate(Sum('price'))

At a glance, it's much easier to understand what's happening in the Eloquent code. Either way, both frameworks handle DB queries quite well and provide enough tooling around DB modeling to suit just about any use case.

Templating

In Laravel the main template engine is called Blade. Blade is fairly easy to understand:

@if(x == y)
// do something
@else
// do someting else
@endif

My {{$variable}} name.

@foreach ($users as $user)
    <p>This is user {{ $user->id }}</p>
@endforeach

@include("somefolder.some_blade_file")

Templates are stored in "resources/views" and in your controller, you can reference a template using the path relative to the "views" folder:

function doSomething() {
    return view("users.index", [$user => $this->user]);
    // Looks for the template in: resources/views/users/index.blade.php
}

Laravel supports both regular PHP files and blade templates, you can use them interchangeably, therefore if you place a file called "index.php" in the user's folder instead, the above code will work just fine as well.

Furthermore, you can also embed PHP code in Blade as well:

@php $x = 1; @endphp
// or
<?php $x = 1;?>

Blade is quite powerful and flexible, in fact, I think it's probably the best template engine I have ever worked with! You can Learn more here, the official Laravel docs are always an excellent resource as well.

Besides Blade, Laravel also supports Intertia, which allows you to write your templates in React and other front-end frameworks.

If you don't want to deal with JS/TypeScript but want interactivity in your views, Blade also can be paired with Livewire. Livewire allows you to build React-like views using just PHP, absolutely powerful and easy to use as well!

Finally, Blade can also be extended using custom "Directives" as follows:

// In class: App\Providers\AppServiceProvider.php

Blade::directive('printFriendlyPostDate', function ($pub_date) {
    return "<?php echo date('j F Y', strtotime($pub_date)); ?>";
});

// Use in your template
@printFriendlyPostDate($post->pub_date)

Django templating looks very similar to Blade:

{% if user.is_authenticated %}
    Hello, {{ user.username }}.
{% endif %}

Hi, My name is {{user.name}}

<ul>
{% for link in links_list %}
    <li><a href="{{link.url}}">{{link.name}}</a></i>
{% endfor %}
</ul>

The major difference between the two is that there is no default convention as to where you should put templates, usually, Django developers place templates in a directory called "templates/" but this is configurable.

To be fair, you can also change Blade's default templates directory, but nobody ever does.

In Django to render a template:

def index(request):
    users = Users.objects.order_by("-register_date")[:5]
    context = {"users": users}
    return render(request, "users/index.html", context)

Notice, Django is much more strict, you have to stick to the template syntax and cannot just use regular Python, however, there is a concept of "template tags" which allow you to write your own custom template functions.

Template tags are similar to the Blade directives mentioned above:

# Place in users/templatetags/custom_tags.py
from django import template
register = template.Library()

@register.filter(name='lower_me')
def lower_me(value):
    return value.lower()

# In your template
templates/users/index.html

{% load custom_tags %}

Hi {{name | lower_me}}

Template tags should be placed in a folder called "templatetags" inside your app folder.

While inertia is optimized for Laravel, it's possible to also use this third-party open-source adapter to get inertia support in Django. I haven't used it myself so can't say if it is any good, seems very basic compared to the Laravel integration.

If you looking for an alternative to Livewire, there is Django Unicorn. I briefly played around with it, but it lacks a lot of features and is very basic, so probably better just to use HTMLX or React.

Either way, Django's support for these kinds of frontend templating is not great, however, Django templates are super powerful and you can always manually add your own integration with React or Vue or any other frontend stack.

Django is strict!

Both frameworks are very opinionated, thus there is always the "Laravel way" or "Django way" of doing things.

Django however is a tad bit more strict and rigid. One example is accessing Auth information:

Auth::user()

In Laravel, you can access this Facade just about anywhere in your code, not just in controllers. In Django on the other hand, you have to pass the "request" object down the line manually otherwise other modules in your code will not have access to the user information.

Generally, facades work in this fashion, where you can simply just import the facade anywhere and access any information that the facade has access to.

This flexibility is great! but not always a good thing when it comes to clean code and even security, so Django's rigidity is probably a blessing in disguise. I have seen some weird projects butchering Laravel, whereas the average Django project seems much cleaner and well-structured.

Background jobs

Both frameworks provide a mechanism for handling background jobs, however, Django is a bit more complicated since you have to set up celery and do some configuration in your app to set up queues.

There are too many steps to cover in this article, but you can read further about how to set up Celery here.

In Laravel to create a background job:

php artisan make:job SendEmails

This adds a new class to "app/Jobs/SendEmails.php". Anytime you need to queue a job, simply invoke the class as follows:

SendEmails::dispatch($some_object_or_data);

// OR to push on any other queue besides the default one:
SendEmails::dispatch($some_object_or_data)->onQueue("myqueue");

This will now store jobs in Redis or your SQL DB and process them in the background (via a "default" queue). You will just need to set up one daemon process:

php artisan queue:work
// you can pass --name to specify which queue if you have more than one.

// Or you can use Horizon if you have multiple queues
// Install horizon first before running: 

// --> composer require laravel/horizon
// --> php artisan horizon:install
php artisan horizon

Horizon is an optional tool you can install, it basically gives you a metrics dashboard and will automatically run and manage all your queues instead of manually setting up a daemon for each queue.

Routing

Both frameworks provide a similar routing mechanism, in Django routes are maintained in a "urls.py" file inside the project folder:

from django.urls import path, include
from accounts import views

urlpatterns = [
    path("user/profile/", views.my_account),
    path("user/<int:id>/", views.view_user),
    path("users/", views.user_list),

    # Group all these URLs with the "admin" prefix
    path('admin/', include('some_app_level_urls.admin_urls')),

    # Named routes
    path('dashboard/', views.dashboard, name="dashboard")
]

Learn more about Django URLs here.

In Laravel, there are "routes/web.php" and "routes/api.php" :

Route::get('/search', 'SearchController@index');

# Named routes
Route::post('/searchTerm', 'SearchController@save')
    ->name("save_search_term");

# Group routes with a prefix
Route::prefix('stores')->group(function() {
    Route::get('/{id}/show', 'StoresController@show');
    Route::get('/{id}/products', 'StoresController@products');
});

Learn more about Laravel routes here. Both routers are fairly flexible and easy to use, Laravel does have a little edge since the syntax is declarative using HTTP verbs and you can easily build routes by chaining methods "prefix", "group", "middleware" and more.

Conclusion

Django in terms of design, clean code, and using the HMVT pattern is probably the better-structured framework, plus Python is really easy to work with, easy to learn, and has a vast ecosystem of libraries for just about anything.

Laravel is not just a framework, it’s an entire eco-system with loads of free and paid first-party packages such as Reverb, Pulse, Horizon, Jetstream, Herd, etc…

Thus Laravel’s ecosystem is more complete and will get you up and running fast regardless of the task at hand. Django on the other hand does have batteries included but these are limited and move slowly.

An example is the Django admin, it still looks like an admin panel from the early 2000s and is a bit cumbersome to customize, Laravel on the other hand has many options to choose from and they each look modern with dark theme support and tailwind.

Furthermore, PHP is still one of the most dominant web languages around and constantly evolving to keep up with modern standards.

All in all, both frameworks are a solid choice. If you prefer the “C” style syntax, go with Laravel, if you prefer Python then Django is your best bet.

If you need modern TypeScript and JS framework support, Laravel is probably the better option with Intertia JS and much better tooling, including Livewire.

This is where Golang shines; it’s a compiled language which means it’s much faster for most tasks compared to Python. Although, similar to Python, it’s fairly easy to learn. The language promotes minimalism and does not include unnecessary bloat even though the standard library is really comprehensive and you often don’t need to reach for third-party libraries.

In this guide, I will give you a general introduction to Golang from the perspective of a Python developer and show you why Golang should be the next language you pick up.

Golang is compiled

This may seem quite obvious, but it makes a huge difference in having code compiled versus being interpreted. Besides just compiler optimizations that can improve the performance of your code, compiled code also ensures types are enforced correctly, thus reducing the number of possible bugs that land up in production (Off-course unit testing is also a good idea).

Python is both strongly typed and dynamically typed. Take this example for instance:

def doMath(x, y):
    return x+y

If you had to run this code, Python will not throw any errors, however, try doing this:

def doMath(x, y):
    return x+y

print(doMath(1, "1"))

You should see:

return x+y
           ~^~
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Without unit tests or static analysis in your IDE, Python code can create many runtime errors if you are not careful. In Golang, it's much harder to create these sorts of bugs because the program won't even compile, even if you are not calling that function or block of code.

Golang variables, arrays, maps

In Golang there are two main ways to create variables:

var someInteger int
# OR
someInteger := 10

You can explicitly set the type using "var" or you can let Go figure out the type by using this operator: ":=". Either way, once you have declared the variable, you can not change its type.

In Python, the following will work fine, but Golang will throw a type error:

x = 1
x = "Some string"

Similar to Python, Golang has a couple of container types:

# Python List: items = [1,2,3,4]

# Golang slice
items := []int{1,2,3,4}

#Python dictionary: carData = {"make": "Toyota", "model": "Corolla"}

# Golang maps
carData := make(map[string]string)
carData["make"]  = "Toyota"
carData["model"] = "Corolla"

In Golang there is no "None" type, however you can simply use "nil" instead. Here are some variable types:

var number int
var number int64
var number float64
var text string
... and most of the common types supported by most languages

var anything interface{}

An "interface{}" type is a special type that allows you to store any other type in that variable. It does not follow the same rules as regular types, it's a dynamic type, thus the compiler will not throw type errors for this type if you do something like:

    var anything interface{}
    anything = "Some text"
    anything = 101
    fmt.Println(anything)

This is terrible Go code and you should avoid using the interface type as far as possible. One common use case for the interface type is handling JSON objects, since JSON can be loosely structured where fields could potentially be a number or text or even an object depending on some condition, you can use this type for these dynamic fields so that the JSON is parsed without throwing type errors.

Here's an example of a dynamic JSON object:

{
   "product_name": "D-T42D15 42 Dled Television",
   "sku": "X42310555",
   "price": 2999.00,
   "currency": "ZAR"
}

{
   "product_name": "D-T42D15 42 Dled Television",
   "sku": 18745534,
   "price": 2999.00,
   "currency": "ZAR"
}

As you can see "sku" is a string sometimes and other times a number. This will cause typing errors in Golang because the type has to be a number or string, and cannot be both.

Therefore you can initially use an interface, and then in your code figure out what the current items data type is at runtime:

// For now, lets assume this function returns the
// product in the form of a struct (similar to python objects).
// We'll learn about structs in a bit...
product := getTheProduct()

// check what the type is and then create a new variable with
// the correct type.
theSKU := ""
switch product.SKU.(type) {
    case string:
       theSKU = product.SKU.(string)
    case int:
        theSKU = fmt.Sprintf("%d", product.SKU)
}

fmt.Println(unpackedValue)

Golang loops, if, switch, and range

Golang is very minimal, thus the language doesn't have as many reserved keywords as Python. For example, the "while" keyword does not exist! Instead, Golang's for loop is very flexible and can be used for a wide variety of loops that other languages generally separate into different reserved keywords.

Here are some examples:

start := 1
end := 10

for start < end {
    start += 1
    fmt.Println(start)
}

// this is the most common Loop. Use when you want to iterate
// over lists or maps.
items := []int{1,2,3,4}
for k, v := range items {
   fmt.Println(k,v)
}

sum := 0
for i := 1; i < 5; i++ {
    sum += i
}
fmt.Println(sum)

// GORM db rows: https://gorm.io/docs/sql_builder.html
rows := db.Rows()

for rows.Next() {
   var name string
    rows.Scan(&name)
    fmt.Println(name)
}

"If" statements are very similar to Python:

age := 22

if age < 18 {
   fmt.Println(num, "You're still a kid.")
} else if age > 18 {
   fmt.Println(num, "You can both drink, and drive. 
     Just not at the same time!")
} else {
   fmt.Println(num, "Sorry, but your youth is slowly fading :-(")
}

In early versions of Python (I think before v3.10), there were no switch statements, however, nowadays you can use "match" and "case". Luckily for Go developers, these exist and work similar to most other modern languages:

age := 22

switch age {
    case 18:
        fmt.Println("You're on your way to adulthood.")
    case 65:
        fmt.Println("#Retirement")
}

In Golang the "break" keyword is not required, Go will automatically break out of the switch on the first positive match. The "break" statement is usually used if your code inside the case block has an IF statement or some other logic that can take multiple pathways.

Golang modules

Modules are essential in any language; In Python creating modules is a breeze, you just use the directory and file name in your import statement to reference whatever code you need:

myapp/
   accounts/models.py

# Becomes
from accounts.models import User

Golang has a similar mechanism, however, it's a tad bit more complicated. The Go file name is usually irrelevant, instead in Go you need to declare a "package" directive at the top of every code file. To start a new Go project, you first have to run the following in your terminal inside the projects root folder:

go mod init mycompany.com/ecommerce

The above will create a "go.mod" file that is essentially your project's package information similar to NPM's "package.json" file. This file will also keep track of all the various dependencies you add to your project.

Next, in every folder where you add a ".go" file, you need to add a "package" directive at the top of the file that's the same name as the parent folder:

 myapp/
   accounts/models.go
main.go # the package for main.go will always be "package main"

Inside models.go:

package accounts

import "fmt"

func PrintHelloWorld() {
    fmt.Println("Hello World")
}

To import this function in our main.go file:

package main
import "mycompany.com/ecommerce/accounts"

func main() {
   accounts.PrintHelloWorld()
}

Your function or variable inside these modules must start with a capital letter to be visible outside of the module, thus if I named the function "printHelloWorld", Go will throw an "undefined" compile error in our main function.

ℹ️ If you created another file inside the module folder "accounts" e.g. utils.go with the "package accounts" directive at the top, that code file will still have access to "printHelloWorld".

To compile a go program you can run the following in your terminal:

go run main.go

# Or if you just want to build the binary without running
 go build -o myapp

Packages can also be imported from GitHub:

package main
import "github.com/labstack/echo/v4"

If you have multiple imports, you should group them as follows:

import (
  "github.com/labstack/echo/v4"
  "github.com/labstack/echo/v4/middleware"
  "net/http" # These style packages are from the standard library.
)

You can also provide an alias for a package if the name is too long to type or if you want to prevent collisions between packages with the same name:

import (
  "github.com/labstack/echo/v4"
  "github.com/labstack/echo/v4/middleware"
  comMiddleware "mycompany.com/ecommerce/middleware"
  "net/http"
)

When you add a git URL like the above to your import, you need to run the following go mod command before building or running the program:

go mod tidy

This is similar to pip, however, you do not need to have a "requirements.txt" file or need to specify each dependency individually. Go's package manager will automatically scan all your code files, look up the URL for that package, and download the relevant code.

Dependencies! What dependencies?

One common issue with Python is system libraries; when you run "pip install xyz", this just installs the Python library but you may still need to manage the system libraries (and also have Python installed on the target machine) via APT or whatever software installer is used by the target machine.

Golang on the other hand, usually does not need any dependencies installed on the target machine, you don't even need the Go compiler installed. Go will automatically generate a self-contained binary that contains everything you need to run the application.

If you have static assets like images or HTML templates, you would need to copy those as well to the target machine, however, Go also provides an embedder that can package everything including static assets into a single binary file.

When building Go binaries you can also specify what machine architecture you want to target via a simple ENV:

GOOS=windows go build

OOP is not a thing in Go

Golang does not work like other OOP languages, there are no classes and objects. Instead, Go has a nifty little "container" type called "structs". They are very similar to classes but much more lightweight.

Here's an example of a basic “struct”:

package main

package main

import "fmt"

type Product struct {
    ID       int64
    Name     string
    Sku      string
    Category string
    Price    float64
}

func main() {
    product := Product{
        ID:       123,
        Name:     "iPhone 14 pro",
        Sku:      "X41233",
        Price:    899,
        Category: "Smartphones",
    }

    fmt.Println(product)
}

ℹ️ Similar to modules, you need to capitalize the first letter of the struct field name otherwise that field will not be usuable on "instances" of the struct.

You can also assign values to struct "instance" one at a time:

    product := Product{}
    product.ID = 123
    product.Name = "iPhone 14 pro"
    product.Sku = "X41233"
    product.Price = 899
    product.Category = "Smartphones"

I keep using quotes around the word instances to emphasize that these are not objects, structs are just containers or collections, they are meant to be lean and efficient.

Structs can also have functions attached to them similar to methods in a class:

package main

import "fmt"

type Product struct {
    ID       int64
    Name     string
    Sku      string
    Category string
    Price    float64
}

func (p *Product) getPrice() float64 {
    return p.Price
}

func (p *Product) updatePrice(newPrice float64) {
    p.Price = newPrice
}

func main() {
    product := Product{}
    product.ID = 123
    product.Name = "iPhone 14 pro"
    product.Sku = "X41233"
    product.Price = 899
    product.Category = "Smartphones"

    product.updatePrice(999)
    fmt.Println(product.getPrice())
}

Note: The same rules apply with function names, if the first letter is not capitalized, the function will not be visible outside of the module it is declared in.

As you build more Go programs, you'll realize that conventional OOP is overrated and Go's minimal approach with structs just makes your code much cleaner and easier to maintain.

You'll notice I skipped past one piece of this code, the "*" character in front of "Product". This essentially is a pointer, instead of copying the product "instance" every time, we pass around a memory location to the original "instance".

Everything in Python is an object, thus the language is already doing this under the hood (passing by reference) so you never need to think about pointers in Python. Golang is just a bit more explicit allowing you to fine-grade control to optimize your code for maximum efficiency.

You can learn more about pointers in Golang and when to use them here.

Concurrency

In Python, "asyncio" can be used to achieve concurrency and speed up your program's execution time. It works great but has a little bit of a learning curve, not too complicated but not as natural as Go's goroutines.

In Golang, you can simply do this:

package main

import "time"

func doSomeWork() {
    time.Sleep(time.Second * 5)
}

func main() {
    go doSomeWork()
}

How simple is that? Just add "go" in front of any function call to make that task run in the background.

Okay, but just one caveat here! If your main program exits before the goroutine finishes, the function will be terminated prematurely. However, usually in Go, your main program would be some kind of background service like an HTTP server, in which case this is not an issue, since the server is always running.

An example HTTP server:

package main

import (
    "fmt"
    "log"
    "net/http"
)

func doSomeWork() {
    fmt.Println("Hello world from goroutine")
}

func main() {
    http.HandleFunc("/api/endpoint", func(w http.ResponseWriter, 
    r *http.Request) {
        go doSomeWork()
        fmt.Fprintf(w, "Hello, World")
    })

    log.Fatal(http.ListenAndServe(":8080", nil))
}

Since "http.ListenAndServe" basically blocks the program from exiting and keeps it in memory forever (unless of course, some fatal error kills the program), you can safely run the goroutine with minimal risk of it not finishing.

If you are running a program that will exit once it finishes all tasks, like a scraper, for example, you will need to stop and wait for all goroutines to finish. You can do this using wait groups:

import (
    "fmt"
    "sync"
)

func doSomeWork(wg *sync.WaitGroup) {
    defer wg.Done()
    fmt.Println("Hello world from goroutine")
}

func main() {
    var wg sync.WaitGroup
    for i := 1; i < 5; i++ {
        wg.Add(1)
        go doSomeWork(&wg)
    }

    wg.Wait()
}

In the above code sample, we first create a wait group:

var wg sync.WaitGroup

Next, in our loop, we "tell" our wait group how many processes we are going to add to the queue, in this case, we just run one goroutine but you can run several more depending on your needs.

wg.Add(1)
go doSomeWork(&wg)

Notice that we also pass "&wg" to the "doSomeWork" function. This is essentially passing the "wait group" by reference so that there is only one "instance" of the "wait group" regardless of how many goroutines we run.

See the power of pointers? We are maintaining a singleton, and thus ensuring the counter is always up to date and accurately matches the number of goroutines we spin up.

func doSomeWork(wg *sync.WaitGroup) {
    defer wg.Done()
    fmt.Println("Hello world from goroutine")
}

In the "doSomeWork" function as expected, we accept a pointer to the "wait group". If you look closely we use "defer" to tell the "wait group" that the function is complete and it should update the counter accordingly.

The "defer" keyword in Golang simply ensures that this line of code is always executed last just before the function finishes, regardless of where you call "defer" in your code.

You can have only one "defer" per function, however, "defer" can also run more than one line of code. Simply wrap your code in a nested function:

func doSomeWork(wg *sync.WaitGroup) {
    defer func() {
      wg.Done()
      fmt.Println("I am done")
    }() // <-- We use () to actually execute the function as
       //      soon as defer is executed.

    fmt.Println("Hello world from goroutine")
}

Coming back to our main code, we call "wg.Wait()" at the end of the main function to stop the program from exiting and wait until all goroutines finish.

func main() {
    var wg sync.WaitGroup
    for i := 1; i < 5; i++ {
        wg.Add(1)
        go doSomeWork(&wg)
    }

    wg.Wait()  // <--------------
}

Handling errors

package main

import (
    "fmt"
    "strconv"
)

func main() {
    var price float64
    price, err := strconv.ParseFloat("hello", 64)
    fmt.Println(price, err)
}

In most languages, you would expect this program to crash with some kind of "NaN" exception, in Python usually you would wrap this sort of logic in a "try...except" block so that you can safely catch the error and gracefully handle the error without crashing the whole program.

Go, on the other hand, does not have a "try...except" block, instead when an error occurs the function simply returns a struct containing more details about the error.

if "err" is nil, then your code executed fine without any errors and you can continue as normal. This may seem odd, but it's actually quite a nice way of handling errors and it keeps your code clean.

You can even choose to ignore the error if you want (although this is usually a bad idea):

package main

import (
    "fmt"
    "strconv"
)

func main() {
    var price float64
    price, _ := strconv.ParseFloat("hello", 64)
    fmt.Println(price, err)
}

The "_" character used in this fashion discards the return value and can be used with any function return type not just errors.

Should you do something silly like this, Go will just refuse to compile the program:

func main() {
    result := 1 / 0
}

Go can throw serious errors known as "panics" which will crash your program, however, these are not common and usually occur when something bad happens outside of your program, e.g. you have an SSD failure and Go cannot write to disk.

Dealing with this kind of error is beyond the scope of this article, but I will cover recovering from serious errors in an upcoming article where we'll take an even deeper dive into the land of Go.

Golang pointers

In the Python world, we are used to passing data by reference since everything in Python is an Object. Thus, you don’t really think about pointers a whole lot, but in Golang not everything is an object, you often have to explicitly tell the compiler how to pass data around.

What is a pointer anyway? Simply put, a pointer is just a variable that contains the address for another complex structure(like a file, object, or something else that holds a value).

To expand on that idea:

brands_list = None
with open("brands.txt", "r") as f:
    brands_list = f.read()

for product in products:
    parseProduct(product, brands_list)

Let’s just say for some bizarre reason you need to load a list of brands from a text file, and that list is needed by a function “parseProduct” which probably parses a raw product object and stores it in the DB. During that process, it needs to access the brands list.

Let’s say we have 1000 products, if we had to read this file at each iteration, that’s going to be inefficient and slow because there’s the disk IO and also the operation of constantly copying 5GB to RAM X 1000. A better approach is to read the file once and then pass the data to “parseProduct”.

Luckily, Python passes by reference, so the memory hit is only 5GB. If Python didn’t pass by reference, you now will have each copy holding 5GB of memory in RAM until garbage collection kicks in.

Golang, on the other hand, will not pass by reference by default:

type Product struct {
    Name string
    // other fields
}

func parseProduct(product Product, brands string) {
    // do stuff with product and brands
}

func main() {
    products := getProducts()

    // ReadFile is a custom function used to keep this code less verbose.
    brands := ReadFile("brands.txt")


    for _, product := range products {
        parseProduct(product, brands)
    }
}

At each iteration, Golang is essentially copying the contents of brands and passing it to the “parseProduct” function, so that’s potentially 10GB of RAM being used, more than double what Python used.

To fix this:

func parseProduct(product Product, brands *string) {
    // do stuff with product and brands
}

func main() {
    products := getProducts()

    // ReadFile is a custom function used to keep this code less verbose.
    brands := ReadFile("brands.txt")


    for _, product := range products {
        parseProduct(product, &brands)
    }
}

Whenever you put a “*” in front of a type in Golang, that makes the variable a pointer. This variable therefore is not actually a string, it’s a memory address of data that is of a “string” type.

Now, we only have one copy of “brands” in memory, and every time “parseProduct” runs, it simply references the initial variable we created(“brands”) instead of repetitively copying the entire file:

    brands := ReadFile("brands.txt")

The “&” basically gets the memory address of the “brands” variable and passes it to the “parseProduct” function, whereas “*” is just declaring that this variable is a pointer of type “string”.

⚠️ When using concurrency, mutating data on a pointer can lead to memory leaks and other race conditions. This topic is beyond the scope of the current article, but, if you want to learn more, I did cover this subject in an earlier article here.

When to use pointers?

This is a topic on its own, but here are some of the main use cases for pointers:

• To maintain a singleton. For example, in our “parseProduct” function we are probably updating a PostgreSQL database table, it’s going to be really inefficient to open and close a DB connection 1000 times. Instead, we can open the DB connection once before the loop and then pass the pointer to that connection so that it can be reused.

• Memory optimization as per the example covered above.

• Mutability: Throughout the process, you may mutate data several times. It can become really cumbersome to constantly ensure you return the mutated version, thus using a pointer will make your code a lot cleaner even if there are no major memory optimization advantages.

package main

import "fmt"

type Product struct {
    Name     string
    Brand    string
    Category string
}

func (p *Product) setBrand(brandName string) {
    p.Brand = brandName
}

func (p *Product) setCategory(CategoryName string) {
    p.Category = CategoryName
}

func parseProduct(product *Product, brands *string) {
    // do stuff with product and brands

    product.setBrand("Some brand")
    product.setCategory("Some Category")
}

func main() {
    products := []Product{}
    products = append(products, Product{
        Name: "Iphone 14",
    })
    brands := ""

    for _, product := range products {
        parseProduct(&product, &brands)

        fmt.Println(product.Brand, product.Category)
    }
}

And here’s the none-pointer version, still valid code but a tad bit more annoying to constantly return “product”:

package main

import "fmt"

type Product struct {
    Name     string
    Brand    string
    Category string
}

func (p Product) setBrand(brandName string) Product {
    p.Brand = brandName
    return p
}

func (p Product) setCategory(categoryName string) Product {
    p.Category = categoryName
    return p
}

func parseProduct(product Product, brands *string) Product {
    // do stuff with product and brands

    product = product.setBrand("Some brand")
    product = product.setCategory("Some Category")

    return product
}

func main() {
    products := []Product{}
    products = append(products, Product{
        Name: "Iphone 14",
    })
    brands := ""

    for _, product := range products {
        updatedProduct := parseProduct(product, &brands)

        fmt.Println(updatedProduct.Brand, updatedProduct.Category)
    }
}

Further learning resources

If you find Golang interesting and want to expand your knowledge further, here are some excellent resources to help you:

• The official Golang docs are an excellent place to start.

• Once you’ve mastered the basics, Effective Go is another essential guide I highly recommend.

• If you are more of a visual learner, I recommend looking at Anything GG’s YouTube channel. He’s a highly experienced developer and has a wide variety of content for both beginners and experienced devs too.

Conclusion

Whoa! I have so much to share but this article has gotten way too long already. Please consider subscribing to my newsletter and you'll get notified as soon as the next article drops where we go deeper into Golang and learn more advanced concepts.

Golang is a beautiful well-designed language, and as you've seen, it's fairly English-like with very little noise, making it an ideal language to grow into coming from a Python background.

Hopefully, you've learned a thing or two from this article and will keep exploring Go further. Happy coding!

💡 Check out my newest article on how to build a WebRTC phone in Python here. With WebRTC, you no longer need PyVoip or a SIP provider, accept calls directly from the browser.

In this article, I am going to walk you through a step-by-step guide on how to build a basic SIP phone and connect that to AI, so that you can have a 2-way conversation with any LLM.

What is SIP Anyway?

Similar to how we have HTTP for the web, voice-over-internet systems usually run on a protocol named SIP, which provides guidelines on how to establish, modify, and terminate sessions over the network.

A SIP session can then carry voice, text, or even video. In the case of our application, SIP is just a signaling protocol and, therefore, is responsible for connecting and disconnecting the call to our Python script.

Once the call is answered and established, we then use the "RTP" or Real-time Transport Protocol to handle the audio stream.

Thankfully with PyVoIP, the library takes care of all the SIP and streaming mechanics, thus we don't have to worry too much about how SIP works or RTP for that matter.

Let's build something cool!?

💡 Are you new to Python? Learn Python with LearnPython.com courses. Fun interactive courses that are based on real-life business scenarios, meaning you’ll be writing Python code and seeing the results instantly. No need to install Python or other tools on your device, everything happens through your favorite web browser (Sponsored content).

In this guide, I will show you how to build a simple phone answering service with Python.

The script will do the following:

1. Register as a SIP VOIP phone and wait for calls.

2. Accept incoming calls.

3. Transcribe the audio using OpenAI's Whisper.

Installing pip packages

We are going to need a few PIP packages as follows:

pip install pyVoIP
pip install pywav
pip install openai

Be sure to also add your OpenAI key to your environment, in bash you can easily do this by doing the following:

nano ~/.bashrc

# Add to the end of the file
export OPENAI_API_KEY="sk-xxx"

You will need to restart your terminal for this to take effect.

Setting up a VOIP virtual phone

PyVoIP is a nifty little library that can easily help you set up a virtual phone with just a few lines of code.

ℹ️ You probably want to use something like Twilio instead for a real-world application. PyVoIP audio quality isn't the best and needs quite a bit of modification to work correctly.

To get started, let's set up a basic phone:

from pyVoIP.VoIP import VoIPPhone, CallState

def answer(call):
    try:
        call.answer()

    except Exception as e:
        print(e)
    finally:
       call.hangup()

vp = VoIPPhone(
    'sip domain', 5060, 'sipuser', 
    'sippassword', callCallback=answer
)
vp.start()
print(vp._status)
input("Press any key to exit the VOIP phone session.")
vp.stop()

In this example, we create a virtual phone using the "VoiPPhone" class. This class takes in a few arguments as follows:

• SIP Credentials: When you purchase a SIP account from a VOIP provider, you should have received a username, password, and an IP or domain name that will be connected to a phone number. (3Cx.com is an example of a SIP provider).

• callCallback: This is the function that will handle answering the phone call.

The callback function will receive one argument, i.e. the "call" object which will contain all the relevant information relating to the caller and provide various methods for you to accept and receive or send audio back to the caller.

ℹ️ Did you know that you can build your own VOIP server as well? Asterisk is a powerful open-source VOIP server that you can use to set up your own SIP accounts, phone numbers, extensions, and so forth.

Transcribing audio

To convert audio into text we can use OpenAI's Whisper service, here's a simple example of how to convert our audio into text:

from openai import OpenAI
import pywav

def convert_to_wav(audio, tmpFileName):
    data_bytes = b"".join(audio)
    wave_write = pywav.WavWrite(tmpFileName, 1, 8000, 8, 7)
    wave_write.write(data_bytes)
    wave_write.close()

    return open(tmpFileName, "rb")

def transcribe_to_text(audio_file):
    tmpFileName = f"/tmp/audio/_audio_buffer_{uuid.uuid4()}.wav"
    client = OpenAI()

    transcription = client.audio.transcriptions.create(
        model="whisper-1",
        file=convert_to_wav(audio_file, tmpFileName)
    )

    try:
        return transcription.text
    except Exception as ex:
       print(ex)

    return ""

The "transcribe_to_text" function takes in a list of raw audio byte samples, we then need to convert those samples into an actual audio file because the OpenAI SDK is expecting a file object, not raw audio bytes.

We therefore use "pywav" in our "convert_to_wav" function to convert the raw audio bytes into a ".wav" audio file.

⚠️ This logic is simplified so that it's easier to understand, but essentially it can be optimized to remove the need for saving to a temp file since disk IO on a slow drive might cause issues.

Updating our answer method to chunk the audio

In our "answer" method we receive the audio as a continuous stream of bytes, therefore each chunk of audio is 20ms. We cannot send a 20ms chunk of audio to Whisper because the minimum length is 100ms.

Thus, we need to append the audio to a buffer and we'll only send the audio to Whisper once we reach 1000ms (or 1 second).

Here is the updated "answer" function:

def answer(call):
    try:
        call.answer()
        buffer = []
        buff_length = 0

        while call.state == CallState.ANSWERED:
            audio = call.read_audio()
            # We divide by 8 because the audio sample rate is 8000 Hz
            buff_length += len(audio) / 8 # or simply 20

            if buff_length <= 1000:
                buffer.append(audio)
            else:
                print(transcribe_to_text(buffer))
                buffer = []
                buff_length = 0

    except Exception as e:
        print(e)
    finally:
       call.hangup()

💡 You can also send back audio to the caller by calling "call.write_audio(raw_audio_bytes_here)"

The full code

from pyVoIP.VoIP import VoIPPhone, CallState
import uuid
from openai import OpenAI
import os
import pywav

def convert_to_wav(audio, tmpFileName):
    data_bytes = b"".join(audio)
    wave_write = pywav.WavWrite(tmpFileName, 1, 8000, 8, 7)
    wave_write.write(data_bytes)
    wave_write.close()

    return open(tmpFileName, "rb")

def transcribe_to_text(audio_file):
    tmpFileName = f"/tmp/audio/_audio_buffer_{uuid.uuid4()}.wav"
    client = OpenAI()

    transcription = client.audio.transcriptions.create(
        model="whisper-1",
        file=convert_to_wav(audio_file, tmpFileName)
    )

    try:
        return transcription.text
    except Exception as ex:
       print(ex)
    return ""

def answer(call):
    try:
        call.answer()
        buffer = []
        buff_length = 0

        while call.state == CallState.ANSWERED:
            audio = call.read_audio()
            buff_length += len(audio) / 8

            if buff_length <= 1000:
                buffer.append(audio)
            else:
                print(transcribe_to_text(buffer))
                buffer = []
                buff_length = 0

    except Exception as e:
        print(e)
    finally:
       call.hangup()

vp = VoIPPhone('xxx', 5060, 'xxx', 'xxx', callCallback=answer)
vp.start()
print(vp._status)
input("Press any key to exist")
vp.stop()

Conclusion

There you have it! A simple phone answering system that can stream live audio and transcribe that audio into text.

As mentioned earlier, PyVoIP is not the best tool for the job since it doesn't handle background noise and static very well. You would need to write some logic to strip out the bad audio samples first before transcribing for an actual real-world application, but hopefully, this is a good start.

In the past 2 years, machine learning has become a vital component of my toolbox as a developer.

Initially, the landscape was wild! We dealt with Keras, PyTorch, Tensorflow, and many other tools. Luckily, since the advent of ChatGPT and Sentence Transformers, it is now easier than ever to build AI-related products.

If you need to sprinkle a little AI magic into your project, you probably can get away with just simple rest calls to OpenAI endpoints, however, if you are building a more complex product and have little to no experience with deep machine learning concepts such as RAG, Langchain and so on, then this article will be an ideal primer for you.

🤦🏾 We actually abuse the term "AI". Machine learning is just one subset of AI. Nonetheless, to keep things simple, since everyone is on the bandwagon of "AI" these days. I will use these terms interchangeably. Sorry!

Models: Give up control

There are no or very few "IF", "THEN", "ELSE", or "DO WHILE" controls in machine learning, thus machine learning is a different paradigm compared to conventional programming.

You will often use natural language and configuration-style programming to tune models in the direction you want them to go. Ultimately though, you never have 100% control over what the model will do. You can In most cases predict the outcome with great certainty but it's not the same as programming where we write rules and control flows to precisely guide the user along the path we want to take them.

A model is largely a "black box" of neural networks. You can think of a neural network as a chain of tiny nodes (neurons). Together all the nodes in a neural network build up some pattern recognition ability that the model uses to analyze users' input and generate the appropriate response.

A neural network usually contains 3 types of layers:

1. The input layer: takes in data as a whole. Neurons in this layer would then represent various features of the input data. For example an image of a "Cat", some neurons will represent the fur, others the tail, others the whiskers, and so forth.

2. Hidden layers: Features extracted from the first layer are analyzed further by these layers to determine weights and biases. Essentially mathematical algorithms are applied to the data, to determine patterns that develop the model's reasoning and prediction abilities.

3. Output layer: This now uses the "learnings" from the previous layer to build an appropriate response back to the user. In the cat image example, we probably want to classify the type of animal. This layer will then return "Cat".

When you train a model, you pass the same data to the model over several iterations. One full data cycle through all layers is known as an "epoch". Each time you cycle through an epoch, the model constantly adjusts and optimizes its pattern recognition ability ( Too many epochs can be bad, read up more on "model loss functions").

Training via epochs is vital to improve the model's accuracy. For example: if the first image is of a "sphynx" cat species, the neural networks are not aware of other cat species yet and therefore will make various assumptions about cats using this species of cat.

This will cause weird predictions since a "sphynx" is significantly different compared to a regular house cat. On the second epoch, the network is now aware of the 99 other cat species and thus has a holistic view of what the different species are and their features. Each subsequent epoch would then re-analyze the data and look for ways to optimize the model's accuracy.

Usually when training a model, you also provide a small sample dataset with accurate examples, so that the model can use this dataset to help fine-tune its accuracy during the training phase.

Tooling and frameworks to build models

In the Python world, PyTorch is the undisputed leader when it comes to building machine learning models. It's open-source and freely available, so just about anyone can implement their own models from scratch, or extend someone else's model.

Before you even build a model, you are going to need some kind of dataset. Usually a CSV or JSON file. You can build your own dataset from scratch using your own data, scrape data from somewhere, or use Kaggle.

💡 Kaggle is a community of "machine learners" where you can find various kinds of freely available datasets and even models to use for both commercial and non-commercial purposes.

Dealing with large datasets in regular Python lists or dictionaries can become inefficient, thus in addition to Pytorch, you will need to learn 2 libraries that assist with parsing and manipulating data (and Langchain for LLM-specific tasks):

• Pandas: This will allow you to load and parse your CSV/JSON training data and format it efficiently into whatever format the PyTorch model needs the data in. Furthermore, Pandas has a consistent API, thus you can easily switch between data sources without needing major refactoring in your scripts.

• Numpy: Models usually work with vector embeddings of your text (numerical representations) which are arrays of floating point numbers. Numpy is used throughout the process by models to perform various numerical computations. Pytorch models in general will return a result in the form of a Numpy array, therefore you will need to process or convert that result accordingly.

• Langchain: A powerful LLM utility library to make working with LLM APIs and Models much easier. You most certainly can use the REST API or Python library for your LLM and work with it just fine without Langchain. For larger applications, Langchain simplifies common tasks, like building system prompts, agents, RAG retrievers, and so on.

A wide variety of pre-trained models

You may be aware of Llama3 and OpenAI models, but in the machine learning world, there are hundreds if not thousands of models available for all kinds of machine learning tasks. Even OpenAI itself has several different models: GPT3.5 (gpt3.5-turbo...) (powers ChatGPT free version), GPT-4 (powers ChatGPT pro), Whisper (for audio to text), DALL·E (image generation) and so forth.

Some of the most popular model types are:

• Text embedding: Converts text into vector embeddings. There is a leaderboard to rank models in this class: https://huggingface.co/spaces/mteb/leaderboard

• Image generation: DALL·E 3, Midjourney, Stable Diffusion, and so forth.

• Text classification: Models that can categorize various pieces of data, like spam classification, product categorization, tagging, etc... Some model examples: FastText, BERT, facebook/bart-large-mnli

• Image classification: Given an image, these models can determine an appropriate label, tag, or caption. Examples: microsoft/resnet-50, OpenAI CLIP.

• Object detection: Given an image, detect various entities in the image. Examples: facebook/detr-resnet-50, keremberke/yolov5m-garbage

• Text to Speech (and vice-versa): microsoft/speecht5_tts, OpenAI Whisper, OpenAI TTS.

• Large language models: General purpose models that can do some or all of the tasks above. Examples: Chat GPT 3.5, Mixtral, LLama3, Gemini.

Similar to "Kaggle" I mentioned earlier, when it comes to models, one of the best places to find pre-trained models is HuggingFace.

In addition to providing models, they also provide an open-source library called "sentence-transformers" allowing you to easily use their models in your code in a consistent way.

Here's a simple example of how you can generate vector embeddings using the Huggingface "sentence-transformers" library:

from sentence_transformers import SentenceTransformer
sentences = ["Some sentence or phrase here"]

# Notice we pass in the model name:
# 'sentence-transformers/all-mpnet-base-v2'
# You can easily swap this model out for any other
# - compatible model e.g. "sentence-transformers/all-MiniLM-L6-v2"
model = SentenceTransformer('sentence-transformers/all-mpnet-base-v2')

# Will convert the sentences into vector embeddings.
embeddings = model.encode(sentences)

# Will print a Numpy array
print(embeddings)

What are vector embeddings?

I have mentioned this concept a few times, so just to clarify for those that are unfamiliar.

Machine learning, under the hood, is essentially statistical algorithms at play, they use a ton of mathematical formulas to generate predictions and perform other machine learning tasks.

Now as you can imagine, text and letters are not mathematical in nature. You can't perform calculations on them, therefore, for the math algorithms to work efficiently, we need to convert words/letters into numbers.

Each phrase or word is then converted into a floating point array of numbers (a vector), and multiple vectors are grouped together (e.g. a sentence or paragraph) to form vector embeddings.

Vector embeddings capture contextual information based on how close words are together, how frequently they appear in similar sentences, and so on. This allows for storing enough information for algorithms like COISINE similarity, ANN, and KNN to calculate semantic meanings of the text.

As you can imagine one of the most useful use cases for vector embeddings is to perform semantic searches. With semantic searches, the algorithm can identify sentences and words even if the search term does not appear in the results (this is different compared to SOLR or Elasticsearch where they just look for synonyms or fuzzy-spelled keywords).

You can generate vector embedding as follows:

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('sentence-transformers/all-mpnet-base-v2')
embeddings = model.encode(["Machine learning"])
print(embeddings)

The result will be a Numpy array (The actual vector embedding is much larger, I have shortened it below to just give you an example) :

[[-2.14947201e-02  6.24050908e-02 -6.70606121e-02  1.17204664e-02
  -2.23315824e-02  3.22391130e-02 -1.10149011e-02  2.08304841e-02
  -9.21188202e-03  3.96048091e-02  1.33278236e-01  4.91726398e-02
  -2.97797378e-02  6.26323521e-02  2.64899582e-02 -8.43470395e-02
   8.12786724e-03  7.97691289e-03 -5.54570891e-02  6.59016613e-03
   1.62357709e-03  7.52743008e-03  2.48725782e-03  4.35999036e-03]]

Depending on the model you use, the dimensions will differ (number of floats). In this case, the model generates "768" floating point numbers. This is not always true, but for the most part, the larger the dimension, the better the accuracy.

What is Retrieval-Augmented Generation (RAG)?

This will be one of the most common machine learning systems you will need to build for web applications.

When you ask ChatGPT a question, it refers back to its vast dataset of data and generates data based on that context. Sometimes it can hallucinate and give you incorrect information.

Other times, it may give you the correct information, but in your unique context that information may be irrelevant or too broad to be useful. For example: "What is the price of a Macbook Pro?", the model responds with "From $1,299....".

This may be correct, but it's not very useful because it doesn't give me the price in my local currency, or provide more detail about each model.

RAG systems aim to fix this problem; essentially what happens, is that you give the model a custom dataset and it then scopes its responses to only your dataset, thus allowing for better accuracy and providing better local context for your domain/business.

This is different from finetuning. In finetuning, you are taking a pre-trained model and extending its ability by providing a custom dataset for your use case. You then have to re-run training on the model and the model will generate a new trained state, thus the model's context is frozen at this state and not real-time.

With a RAG system, you are not re-training the model, you are providing context data in real-time, therefore you do not need to train the model every time the data changes.

The LLM uses its original training data as its basis to develop reasoning, but any generation or prediction tasks it performs using RAG will be scoped to your custom data and precedence will be given to your data instead of its original dataset.

If you want to learn how to build a RAG system, you can have a look at an earlier article I have done here.

How to host your own LLM?

When you first get into machine learning, I would advise you not to tinker with open-source models like Llama3, rather focus on using OpenAI models first and get familiar with their APIs.

While OpenAI credits are fairly cheap and can get you good mileage, sometimes though, when you are building a large-scale application, you may want to run your own in-house model.

You have a few open-source options:

• Llama3 - Developed by Facebook.

• Mixtral - Developed by an Independent company that's making waves in the ML world.

• Phi3 - Developed by Microsoft, very compact and efficient. This model can even run on mobile devices.

I wouldn't touch Google's Gemma model, I had tried it initially and the results were poor, however, to be fair it was within the first week of it being launched.

Depending on your GPU resources, you may want to try Phi3 first, then Mixtral, and finally Llama3. Llama3 should be the best-performing model. It is a tad more resource-intensive and you may not always need that kind of power.

To run your models, you can use ollama. I also did a tutorial on setting this up here.

Conclusion

Hopefully, I have given you enough insights to get started on expanding your skillset into machine learning, of course, machine learning has gone through many changes in recent years, and libraries/models are still constantly evolving.

I would advise not to get caught up in all the hype, rather pick up Pandas, NumPy and play around with Huggingface models first.

Once you have a solid grasp of these, then move on to Langchain and integrate with OpenAI to build a simple RAG, followed by a simple agent.

Happy building!

Qdrant is a great alternative to FAISS. In this article, we will look at building a basic similarity search using Langchain and Qdrant.

What is Qdrant?

In the world of machine learning these days, it's a common practice to vectorize text and store it in a backend DB, unfortunately, most modern databases such as MySQL are not designed to store vector embeddings, although things move fast in the machine learning world and support is getting better, vectors are not yet a first-class citizen for RDBMS type databases.

Qdrant is a modern database that's optimized for storing unstructured data, more especially vector embeddings. Besides being a vector store, you can also use Qdrant as a search engine, it lacks features such as faceting but still is pretty useful on longtail phrase type searches.

Take for example: "Please suggest a high-end gaming machine with lots of RAM and a good graphics card ", a regular keyword search might return no results or poor results.

I tried this on Amazon.com and got weird results for: RAM, mini PCs, and socks 😂:

Naturally, this confuses a keyword-based search engine because it's looking for specific keywords or their synonyms. On Qdrant, you can most probably find "gaming laptops" for the same above search. Since the vector-based search also stores "meaning" and uses algorithms such as "kNN" with cosine similarity to better understand the searcher's intent.

If you don't know what vector embeddings are: they are basically a numerical representation of text, that machine learning models can use to perform math calculations, in-order to determine meaning and relationships between words and phrases. Qdrant's website provides a more in depth article on this subject as well, you can learn more here.

Setting up Qdrant

Similar to most NoSQL databases, Qdrant uses the concept of "collections". Essentially a collection is a unique grouping of documents, similar to an SQL table.

The easiest way to get started with Qdrant, is to use the docker image:

docker run -dit -p 6333:6333 qdrant/qdrant

You should now be able to view the web dashboard by visiting the following link: http://127.0.0.1:6333/dashboard

Before we get to the code, you will need to install a few pip packages:

pip install langchain
pip install qdrant_client
pip install langchain-community
pip install langchain-openai

Setting up our schema:

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams
from langchain.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import Qdrant

embeddings = OpenAIEmbeddings()
client = QdrantClient('http://127.0.0.1:6333')

langchain_qdrant_store = Qdrant(
    embeddings=embeddings,
    client=client,
    collection_name="products"
)

# Should only run this once
client.create_collection(
    collection_name="products",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
)

When you run the above, it should create a collection named: "products" with one field that will house our vector embedding. For the purposes of this article and to keep things simple, I am just storing the vector embeddings and no other extra meta information.

You can most certainly, store more information using "payloads" if needed. Learn more about payloads here.

Storing documents in the collection

Storing documents in the collection is fairly straightforward when using the "OpenAIEmbeddings" and Langchain Documents. Langchain will automatically take care of the complexity of querying OpenAI and generating the vector embeddings.

So basically all you need to do, in order to index a document in Qdrant is as follows:

from langchain.docstore.document import Document

doc = Document(page_content="Some text data here")
langchain_qdrant_store.add_documents([doc])

If you want to store documents like PDF’s, here’s an open-source project I built some time ago that provides an example: kevincoder-co-za/ragable

Finding similar documents

To perform a KNN search we simply need to do the following:

documents = langchain_qdrant_store.similarity_search("Some search term")

The above will return a list of "Langchain documents", which you can pass on to an LLM or process further just like you would any other Python object in your application.

Using the Qdrant store as a retriever

Lanchain is awesome and can simplify the process of passing on this information to an LLM for further reasoning, see the example below:

from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-3.5-turbo-0125", temperature=0)
chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=langchain_qdrant_store.as_retriever()
)
print(chain({"query": question}))

Whenever you pass a question to the LLM, it will automatically query Qdrant for similar documents and then determine its response accordingly based on that context data.

This is quite powerful and is known as "RAG" (retrieval-augmented generation**).** Remember with ChatGPT and other such models, it's often possible for the LLM to hallucinate or provide a response that may be correct but not as in-depth as you may like.

With RAG, you are providing the context data and can scope the LLM to only respond based on the data you provide, this reduces the model's likelihood to hallucinate, and you can enforce an "I don't know" rule or something to that effect when it cannot answer the question effectively.

What can I build with this kind of system?

Some common use cases include:

• A summarization tool. You can vectorize hundreds or thousands of documents into Qdrant and then have the LLM summarize and answer questions based on that data. This is useful in various business applications: such as training, onboarding, customer support, and so forth.

• Insights tool. Ingest large amounts of statistics and then build an interface with various charts to predict trends and understand your data better. Crime intelligence is a good example.

• Suggestion engine. A great use case would be e-commerce, the chatbot can help users with their purchases by providing more information and suggesting other products that may go well together and so forth.

• Similarity search. You don't need a full RAG for this, since it'll be slow and use lots of resources if you have a high volume of searches. Qdrant alone is sufficient though using the "similarity_search" function.

These are just a handful of suggestions, I am sure there are loads more interesting applications that can be built using this platform.

What are the disadvantages of using Qdrant?

One of the major pain points with Qdrant is how the querying engine works, it’s similar to NoSQL but very basic and cumbersome to work with if you need to do some advanced filtering like you would in SQL:

where city="Cape Town" and price >= 500

In Qdrant:

POST /collections/{collection_name}/points/search

{
  "filter": {
    "must": [
      {
        "key": "city",
        "match": {
          "value": "Cape Town"
        }
      },
      {
        "key": "price",
        "range": {
          "gte": 500
        }
      }
    ]
  }
}

As you can see, it’s so much more verbose and cumbersome to work with versus just plain old SQL, however, this is not a problem usually because 90% of the time you basically just use Qdrant for the similarity search and then can get back a list of IDs or something to identify your data and then use those to query your SQL backend.

Another issue is that while their documentation is fairly okay, it’s still very new thus there are very limited resources and you have to dig into the docs quite a bit to find what you looking for(although these days we just ask Claude right?).

In some cases you may need more than just a well-written prompt, you may want to trigger different data sources or actions based on what the user asks.

For example: in an e-commerce site, if the user asks to view a list of "shoes", you probably would do a keyword/semantic search and return a list of matching shoes.

If they ask about delivery information, you may want to look up that data from an SQL DB or API.

Langchain agents are a powerful mechanism at your disposal that will enable you to build complex custom LLM chatbots. In this article, we will go over what is an agent and how to build one.

What is a Langchain Agent?

When you use an LLM like OpenAI's "chatgpt-turbo", you typically will send the model a prompt consisting of one or more messages, and the LLM will respond accordingly.

So essentially text in and text out, this is okay for a chatbot that is performing one particular task like answering support questions, but what if you need to change the data source or do a web search or record something in the DB?

This is where Agents come in handy; an Agent is basically a "task executor". It will allow the LLM to execute functions and other code in your application based on reasoning and user input.

Think in terms of a switch statement, depending on which pathway is true, the switch statement will execute that particular block of code.

Agents are not exclusive to Langchain. Each LLM has a different way of handling agents, however, Langchain just provides a consistent API to work with regardless of which backend LLM you are using.

Tools

Since agents are task executors, we need some kind of "callback" for the agent to execute such as a function or class.

Tools take in, either the raw prompt or a list of arguments and return some sort of output. Usually, you would return a string, but it's also possible to return more complex data like a LangChain document.

There are multiple ways of declaring a tool. We will cover the decorator approach since it's the most common and easiest solution to understand.

Here is an example:

from langchain.tools import tool

@tool
def search_delivery_information(prompt :str) -> str:
    """ When the user requests delivery information """

    # We now return some text from an external API
    # The LLM will analyze this text and 
    # - return the appropriate answer to the user.
    return requests.get("/somewhere/delivery.json").text

Three essential components make up a tool:

1. @tool - This decorator will take care of handling the input/output of your function in a way that the LLM can understand.

2. """ - The docstring; think of this as a prompt system message to the LLM, where you tell the LLM when to execute this function and provide any other useful context data.

3. The returned data. The LLM will ingest your function's response as context data and scope its response to that context.

To clarify what I mean by "scope", without any context data, if the user asks: "Which is the best shoe brand?", it will respond with some generic response based on the LLMs training data, similar to asking ChatGPT a question, so probably it may respond with "Nike" or "Reebok".

However, if the tool returns "Addidas is our best brand.", then the LLM's response will regard "Addidas" as the best brand and not "Nike" or "Reebok".

Putting it all together

Okay great! Now you know what an agent is and how to create your custom callback functions to help the LLM better answer the user's question.

Let's now build an Agent and link it to our custom tool:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.agents.format_scratchpad.openai_tools import (
    format_to_openai_tool_messages,
)
from langchain.agents.output_parsers.openai_tools import OpenAIToolsAgentOutputParser

# Create a standard Chat LLM
llm = ChatOpenAI(model_name="gpt-3.5-turbo-0125", temperature=0)

# Create a list of all tools you want to enable
tools = [search_delivery_information]

# Connect our tools to the LLM
llm_with_tools = llm.bind_tools(tools) 

# Build a chat prompt.
# Notice we have placeholders for the user's input
# - and a second placeholder for the Agent's context data.
prompt = ChatPromptTemplate.from_messages(
        [
            (
                "system",
               f"You are an e-commerce assistant.",
            ),
            ("user", "{input}"),
           MessagesPlaceholder(variable_name="agent_ctx"),
        ]
)

# Next we build the actual agent.
agent = (
        {
            "input": lambda x: x["input"],
            "agent_ctx": lambda x: format_to_openai_tool_messages(
                x["intermediate_steps"]
            )
        }
        | prompt
        | llm_with_tools
        | OpenAIToolsAgentOutputParser()
)

  {
      "input": lambda x: x["input"],
      "agent_ctx": lambda x: format_to_openai_tool_messages(
           x["intermediate_steps"]
      )
  }

The above block might seem confusing at first glance, but basically, the first argument is the user's input, the prompt template created earlier will replace "{input}" with the actual user's question or message.

Secondarily: "agent_ctx", since our tool callback functions are just Python functions, there needs to be a translation step that converts the output from these functions into something that the model can understand and the agent can transmit via the REST API.

You will also notice we chain one other object at the end "OpenAIToolsAgentOutputParser", this will receive data and convert it into a format that the agent can understand.

Finally, we can instantiate an agent executor and prompt the LLM:

from langchain.agents import AgentExecutor

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=False)
result = agent_executor.invoke({"input": question})
print(result)

Wait! What about memory?

Naturally, users will ask follow-up questions and the LLM needs to be aware of these to provide a much more accurate answer.

Luckily, Langchain makes managing memory so much easier. Here is an example:

from myapp.models import Message
from langchain.prompts import MessagesPlaceholder
from langchain_openai import ChatOpenAI
from langchain_core.messages import AIMessage, HumanMessage
from langchain_core.prompts import (
ChatPromptTemplate,
MessagesPlaceholder
)
from langchain.agents.format_scratchpad.openai_tools import (
    format_to_openai_tool_messages,
)
from langchain.agents.output_parsers.openai_tools import (
OpenAIToolsAgentOutputParser
)

# ... other agent code as per above snippets
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            f"You are an e-commerce assistant.",
        ),
        # Add new variable to replace with previous chat messages.
        MessagesPlaceholder(variable_name="chat_history"), 
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_ctx"),
     ]
)

# Build a list of messages from the DB.
chat_history = []

# Very simple query, you may want to limit to
# - the last 10 conversations or something to that effect.
messages = Message.objects.filter(is_deleted=False)
for m in messages:
    if m.model_answer is not None:
        chat_history.append(
            [
                HumanMessage(content=m.user_question),
                AIMessage(content=m.model_answer),
            ]
         )

agent = (
    {
        "input": lambda x: x["input"],
        "agent_ctx": lambda x: format_to_openai_tool_messages(
            x["intermediate_steps"]
        ),
        "chat_history": lambda x: x["chat_history"],
    }
    | prompt
    | llm
    | printer
    | OpenAIToolsAgentOutputParser()
)

In the above code, we first add a variable to the original prompt template: "chat_history" and then in our agent pipeline we extract the messages from the list constructed above in "chat_history".

Security concerns

Since tool functions are essentially serialized and passed to the OpenAI REST API, it's important to ensure you do not expose any sensitive information in these functions.

Furthermore, if you are allowing the LLM access to interact with your SQL RDMS, just be careful of SQL injection. Rather, build an API around your data or use a vector-based database like Qdrant.