Mindfire Technology

When to Call the LLM?

Wed, 27 May 2026 09:12:39 -0600

I've been building a Book-to-Audio pipeline. See, for example, here. The idea is simple: take a PDF or EPUB, parse it into clean paragraphs, and feed those paragraphs to a text-to-speech engine. Simple enough. But scanned books are messy. OCR artifacts like Whenin (a word join from "When in"), hyphenated line breaks carried over from print, footnote markers embedded mid-sentence — the raw text often needs cleaning before it's speakable.

So I added an LLM-based cleaner. You pass it a paragraph as well as page context, it returns a cleaned version and a classification: body, footnote, or drop. It works well. The problem is I was calling it on every paragraph, which took forever using a local LLM.

The Book2Audio repo is found here.

Was That Actually a Problem?

If I was just burning tokens for Claude or ChatGPT, time wouldn't be an issue. But token budget would be.

Either way, a typical chapter has hundreds of paragraphs. The LLM takes roughly five seconds per call. Most paragraphs — the ones that say "He has refused to pass Laws of immediate importance" — are already perfectly clean. Calling the LLM on those is waste, pure and simple, because any 'improvement' from the LLM would necessarily be a departure from what the book actually said. For an audiobook converter, that's not desirable.

The more interesting question is: when do we actually need the LLM? The answer is when the text contains something the parser couldn't fix — OCR artifacts, broken word joins, corrupted characters. Not when it contains the word "justice."

So I built a gate.

The Gate: `_all_words_valid`

The idea is straightforward. Before calling the LLM, check whether every word in the paragraph is already valid English. If they all are, skip the LLM entirely. The paragraph is clean enough.

def _all_words_valid(text: str) -> bool:
    for token in text.split():
        stripped = re.sub(r"[,;:.!?()'\"—–]", '', token.lower())
        if not stripped or not word_validator.is_valid_word(stripped):
            return False
    return True

word_validator.is_valid_word() is doing real work here: it checks the NLTK English words corpus, then falls back to Porter stemming and WordNet lemmatization. So "running", "armies", "endeavoured" — these all pass. Just checking a raw dictionary wouldn't cut it. English inflection is too irregular.

The result: the Declaration of Independence, which has clean prose, now sends only a handful of paragraphs to the LLM — the ones with genuine OCR artifacts — instead of all of them.

What Was Actually Slow?

Once I had the gate in place and added timing debug statements, the results were revealing:

[TIMING] validation=6.24s (1 skipped) | llm=9.26s (1 calls) | total_timed=15.50s

Six seconds on word validation for one paragraph? That can't be right. The validation logic is fast. What was happening is that NLTK loads its word corpus, stemmers, and WordNet lemmatizer lazily — on first use. The first call to is_valid_word() was triggering the entire NLTK initialization chain mid-paragraph, and that takes six seconds.

The fix is a warm-up call at the start of processing:

if self._cleaner is not None:
    word_validator.is_valid_word('warm')

After that, the timing looked quite different:

[TIMING] validation=0.01s (14 skipped) | llm=31.11s (7 calls) | total_timed=31.12s

Validation: effectively free. LLM: the clear bottleneck, as expected.

What the Remaining LLM Calls Reveal

Of the seven paragraphs still going to the LLM, only two were genuine OCR artifacts. The other five were false positives — cases where the validator correctly identified a token it didn't recognise, but the token was actually fine:

self-evident — the hyphen confused the token stripper
endeavoured (twice) — valid British spelling, not in the American NLTK corpus
offences — same problem
compleat — archaic spelling, borderline

This is worth sitting with for a moment. The gate is working correctly in the sense that it's catching tokens it genuinely can't validate. The problem is that "can't validate" and "needs LLM cleaning" are not the same thing. A British spelling is not an OCR artifact. We're using the wrong signal.

A better gate would know about British English. A simpler gate would split hyphenated words and check each part. Neither fix is difficult. I haven't made them yet.

What I'd Do Differently

The timing instrumentation taught me something I should have measured from the start: where is the time actually going? I had assumed the LLM was the bottleneck. It was — but only after fixing a completely separate problem (NLTK lazy loading) that was masking everything else.

This is a mundane lesson dressed up as a profound one: don't optimise until you measure. But there's a sharper version of it. The six-second NLTK startup cost was invisible in normal use because it happened once per run. It only became visible when we added the gate — because the gate called is_valid_word() on the first paragraph instead of waiting for the LLM to warm up later. The optimisation revealed the bottleneck it was supposed to solve.

That's not irony. That's just what measurement does. You don't know what's slow until something makes the slow thing visible.

Where This Leaves Things

The pipeline is meaningfully faster. Most paragraphs skip the LLM entirely. The remaining LLM calls are concentrated on the paragraphs that actually need attention. The test suite now runs in about 30 seconds instead of... considerably longer.

What remains: British spellings, hyphenated words, and the general question of what "valid English" really means for a validator that's supposed to be catching OCR artifacts, not judging orthographic conventions. That's a problem worth thinking about carefully before reaching for a fix.

However, some caution is warranted. The gate is a heuristic. It will miss things. A paragraph full of correctly-spelled words can still be semantically broken in ways no word list will catch. The LLM is still there for the hard cases. That's probably the right architecture — use it as a last resort, not a default.

Lesson's Learned

Of course this would all go a lot faster if I wasn't using a local LLM, in this case llama3.1:8b. That is a future idea. But I wanted to be able to run this locally if needs be.

The idea of a gate to reduce calls to the LLM sped things up quite a bit. The checking if the LLM didn't make a mistake will be a topic for a future post. But let's just say that LLMs can sometimes get creative. So a shorter leash may be necessary. Using NLTK -- the poor thing seems almost defunct since the advent of LLMs -- really does save some time on the verification that the LLM didn't hallucinate. Overall, I'm pleased with the progress.

If you need help with your Artificial Intelligence solutions, we're here to help.

Introduction to Genetic Programming

Wed, 20 May 2026 00:00:00 -0600

Genetic programming addresses the problem of automatic programming, namely, the problem of how to enable a computer to do useful things without instructing it, step by step, on how to do it. (John Koza in Genetic Programming: An Introduction, p. vii)

Genetic Programming is a type of Machine Learning that builds programs via a simulated form of Darwin's evolution by natural selection. Starting with a code base I got from Toby Segaran's excellent book Collective Intelligence (which might be one of the best and briefest books on Machine Learning I've seen) I will layout how Genetic Programming works and then perform a couple of experiments to see if I can improve the starting code base. I hope to go further in future posts.

The actual code I used can be found in this Python Notebook on Google Colaboratory and you can even tweak and run the code yourself.

Darwin's Program

In Genetic Programming we have a hidden function we want to automatically write a program to solve. All of life is a function, so this might be anything: predicting chances of a heart attack, find the best strategy playing a game, etc. For our purposes, we're going to be using a simple hidden function (taken from Segaran's book):

X^2 + 2Y + 3X + 5

What we'll do is write a function to randomly feed various X and Y parameters into that function and get back the correct solution. These results will all be stored in a table. That table will look something like this:

Of course in reality it will have a lot more rows. But the goal of our Genetic Programming algorithm is to build up a population of code that, over generations, gets better and better at predicting the correct result using the data in that table. With some luck, the Genetic Programming algorithm will find the actual hidden function and perfectly predict the results.

Our Toy Programming Language

For this experiment, we're going to stick with a very simple programming model that is made up of the following functions: add, multiply, if, >, subtract. In addition, it will be able to specify (for each of these functions) a parameter or a constant. While this is hardly a complete programming language, it is sufficient to find our hidden function and it should give you a pretty good idea how genetic programming works.

Programming Trees

Now you might be wondering how in the world can we evolve a program? If you're asking this, you probably have in mind typing a program into a text editor or Visual Studio and then randomly throwing statements together. Of course, if you did this, there would be primarily just bad syntax. To avoid this problem, we're going to use a trick where the program has to exist in a tree. So imagine a program that looks like this:

This programming tree is equivalent to our hidden function and would perfectly predict each result without error.

Building a Population

So how do we actually evolve toward a tree like that? We start with a population of randomly generated trees. Most of them will be terrible — they'll wildly mispredict the results in the table. But "terrible" is still useful information. We score each program by running it against the data and measuring how far off its predictions are. This score is called the fitness of the program. Low error = high fitness.

Now here's where Darwin comes in.

We take the fittest programs and use them to breed the next generation. There are two main operations for doing this:

Mutation — Take a program tree and randomly change one of its nodes. Maybe swap a multiply for an add, or replace a constant with a parameter. This is the equivalent of a random genetic mutation.

Crossover — Take two program trees and swap a random subtree between them. This is the equivalent of sexual reproduction — combining the best features of two successful programs in hopes of producing something even better.

Run enough generations of selection, mutation, and crossover, and the population tends to converge on better and better solutions. Sometimes it finds the exact hidden function. Sometimes it finds a different function that happens to produce the same outputs for all the data in the table (a perfectly valid solution, even if it isn't the "original" one).

The Experiment

I ran the algorithm against our hidden function X^2 + 2Y + 3X + 5 with a population of 500 programs over 100 generations. The results were encouraging. Within a few dozen generations the best program in the population had dramatically reduced its error score, and by the end it had converged on a solution that perfectly predicted every row in the table.

You can see the exact results — and run the code yourself — in the Python Notebook. I'd encourage you to play with the population size and generation count and see how it affects both accuracy and speed.

What's Next

This is just the starting point. A few obvious questions worth exploring in future posts: Can we improve the algorithm's speed by being smarter about how we generate the initial population? Does a larger population always produce better results, or does it just produce slower ones? And perhaps most interestingly — how does Genetic Programming hold up against other Machine Learning approaches on harder problems?

All theories, including this algorithm, are only as good as the problems we test them against. So the next step is to make the problem harder.

The No Free Lunch Theorem: Why No Learning Algorithm Is Universally Best

Thu, 14 May 2026 00:00:00 -0600

In my previous post on inductive bias, I ended with an open question: is there an "optimal" inductive bias -- one whose search space is universal but whose search strategy is still efficient and tractable for any problem?

Aren't humans an "optimal" general learner compared to, say, existing machine learning algorithms? So intuitively it seems like the answer should be "yes, there is an optimal learner."

As it turns out, there is a mathematical proof known as the "No Free Lunch Theorem" that proves the answer is actually "no." It is one of the most important results in the theory of optimization.

In 1997, David Wolpert and William Macready published a paper called "No Free Lunch Theorems for Optimization" that proved something remarkable: averaged over all possible problems, no optimization strategy performs better than any other. Including random search. Including random guessing. Including humans in the loop. Every strategy that gains an advantage on some class of problems pays for it with equal disadvantage on another class.

And when they say "strategy," they mean it broadly. As Ho and Pepyne put it in their accessible explanation of the theorem: "Strategies include methods involving search, adaptation, learning, voting, feedback, dynamic programming, evolution, randomization, and even humans in the loop. In short, the concept of strategy covers any method for coming up with a solution to an optimization problem. Nothing can be more general or more inclusive" (Ho and Pepyne, 2001).

This sounds absurd. We know that some algorithms work better than others in practice. How can all strategies be equally good? The answer lies in those three words: "all possible problems." Let me show you why.

A Simple Pathfinding Problem

Imagine a robot that needs to find the shortest path from point A to point D through a small network:

    B
   / \
  A   D
   \ /
    C

There are two possible routes:

Path 1: A → B → D
Path 2: A → C → D

Each path has a total distance. The robot does not know the distances in advance -- it must pick a path and find out. The goal is to pick the shorter one.

The Universe of All Possible Problems

Let us say each path's total distance can be Short (1), Medium (5), or Long (10). A "problem" is a specific assignment of distances to both paths. In the formal framework, each such assignment is a function -- labeled f0, f1, and so on -- that maps each path to a distance. Since each path can independently be 1, 5, or 10, there are 3 x 3 = 9 possible functions. Here they all are:

The P-Matrix: All 9 Possible Problems

Problem:    f0   f1   f2   f3   f4   f5   f6   f7   f8
Path 1:      1    5   10    1    5   10    1    5   10
Path 2:      1    1    1    5    5    5   10   10   10

This table is what Ho and Pepyne call the P-matrix. The rows represent the available choices. The columns represent every possible problem -- every possible assignment of distances to paths. The entries are the distances.

Most of these nine problems will never occur in the real world. Some of them might correspond to a real map. Others are pure mathematical fiction -- worlds where both paths are equally short, or where the path that looks longer on a map is actually shorter. The P-matrix does not care about physical plausibility. It enumerates everything.

Now consider two strategies:

Strategy 1: Always take Path 1.

Strategy 2: Always take Path 2.

Strategy 1 gets the Path 1 distance on every problem: 1, 5, 10, 1, 5, 10, 1, 5, 10. Total: 48.

Strategy 2 gets the Path 2 distance on every problem: 1, 1, 1, 5, 5, 5, 10, 10, 10. Total: 48.

The totals are identical. Averaged over all nine possible problems, neither strategy is better than the other.

Where Each Strategy Wins

The totals are the same, but the individual problems tell a more interesting story.

Strategy 1 wins on f3, f6, and f7 -- problems where Path 1 is shorter than Path 2. Strategy 2 wins on f1, f2, and f5 -- problems where Path 2 is shorter. They tie on f0, f4, and f8.

On the problems where Strategy 1 wins, it wins by a combined total of (5-1) + (10-1) + (10-5) = 18.

On the problems where Strategy 2 wins, it wins by a combined total of (5-1) + (10-1) + (10-5) = 18.

The gains and losses cancel perfectly. This is not a coincidence.

Why the P-Matrix Makes This Inevitable

Look at the P-matrix again. The columns enumerate every possible combination of distances. In the Path 1 row, every possible distance (1, 5, 10) appears exactly three times. The same is true for the Path 2 row.

Ho and Pepyne point out that mathematically this is a counting matrix -- a matrix whose columns count through all possible value assignments. The key property of a counting matrix is that all row sums are equal. You can verify this by inspection: both rows sum to 48. No row can have a higher total than any other when the columns enumerate every possible assignment. This is a mathematical certainty. We showed this for our small 2 x 9 matrix, but the property holds for any size. As long as the columns enumerate every possible combination of values, the matrix is a counting matrix and the row sums will always be equal.

And that is the No Free Lunch theorem. No matter what strategy you use -- no matter how sophisticated, how clever, how well-informed -- if you sum its performance across all possible problems, you get the same total as any other strategy. The P-matrix is a counting matrix, and counting matrices have equal row sums. There is no way around this.

But Real Algorithms Do Work Better

If all strategies are truly equal, why do real algorithms outperform random guessing in practice?

Because real problems are not drawn uniformly from all possible problems. The real world has structure.

Consider the A-star algorithm, one of the most effective pathfinding algorithms ever developed. A-star uses a heuristic to decide which paths to explore first. In our network, if B is geographically close to D, A-star's heuristic estimates that the path through B is likely shorter. It explores that path first.

This heuristic relies on a specific assumption about the world: that geographic proximity correlates with travel distance. In Euclidean space, this is guaranteed. If B is close to D as the crow flies, then the travel distance from B to D cannot be wildly longer than the straight-line distance. This property -- essentially the triangle inequality -- is what makes A-star's heuristic admissible, meaning it never overestimates the true remaining distance.

In the real world, this assumption holds. Roads may be winding, but a point that is one mile away as the crow flies is never a thousand miles away by road. A-star exploits this structure to prune bad paths without exploring them, which is what makes it fast and effective.

But now imagine a world with a trans-dimensional hopper -- a device that warps space so that two points that are far apart as the crow flies can have a travel distance of nearly zero--or even a physically impossible negative distance. In this world, node C might be geographically far from D, but the hopper road from C to D is absurdly short. A-star's heuristic looks at C, estimates a long remaining distance based on the straight-line measurement, and concludes "that direction is not worth exploring." It prunes the path through C -- the path that, thanks to the hopper, is actually the shortest.

A-star does not just miss the optimal path. It confidently excludes it. Its heuristic, which is so reliable in the real world, becomes actively misleading in a world where the relationship between straight-line distance and travel distance is broken. But a random search strategy -- which assigns no meaning to geographic proximity and just tries paths arbitrarily -- would have an equal chance of stumbling onto the hopper path.

This is the No Free Lunch theorem made concrete. A-star's inductive bias is the assumption that geometry is well-behaved. That assumption makes it brilliant in our world and blind in the hopper world. The P-matrix contains both kinds of worlds, and the gains and losses cancel.

The Connection to Neural Networks

The same logic applies to every learning algorithm we have discussed in this series.

Recall from the inductive bias post that Mitchell characterized backpropagation's inductive bias as "smooth interpolation between data points." A neural network trained with backpropagation assumes that the underlying function is smooth -- that nearby inputs produce nearby outputs. This is what allows it to generalize from training data to new examples.

But the NFL theorem tells us: for every smooth function where this assumption helps, there exists an anti-smooth function where it hurts by exactly the same amount. A function where nearby inputs map to wildly different outputs will fool the neural network into confidently predicting smooth transitions that do not exist. On that function, random guessing would do just as well.

The neural network's situation is identical to A-star's. Its inductive bias -- smoothness -- makes it powerful on the kinds of problems the real world actually presents. But that power comes at a cost: poor performance on problems that violate the assumption. The P-matrix contains both kinds, and the row sums are equal.

The Connection to Popper

This is Mitchell's "futility of bias-free learning" -- from our first post in this series -- generalized to all of optimization.

Mitchell showed that a single learning algorithm with no inductive bias cannot generalize at all. The NFL theorem shows something broader: not only do you need an inductive bias to generalize, but no single inductive bias is universally best. Every bias helps on some problems and hurts on others. There is no free lunch.

And this is Karl Popper's point yet again. There is no universal method of discovery. There is no algorithm that works for everything. Every act of learning, every act of optimization, every act of scientific discovery requires prior assumptions about the structure of the problem. Those assumptions are what make progress possible -- but they are also what make us fallible. For some class of problems, they necessarily fail.

The question is never whether your algorithm has assumptions. It always does. The question is whether those assumptions match the world you are actually in.

The No Free Lunch Theorem was first proved by Wolpert and Macready (1997). The P-matrix framework and counting matrix explanation is from Ho and Pepyne (2001). All references to Mitchell are from Machine Learning (McGraw-Hill, 1997).

If you need help with your Artificial Intelligence solutions, we're here to help.

Inside the Qwen3-TTS Engine Code (Qwen3-TTS, Part 2)

Thu, 07 May 2026 00:00:00 -0600

This is the follow-up to my previous post on adding Qwen3-TTS to Book2Audio. That post covered how to use Qwen3-TTS from the command line and how I refactored the code to support multiple TTS engines. This post walks through the actual engine code — how it's structured, what each piece does, and how Qwen3-TTS works under the hood.

All the code discussed here is available in my GitHub repo.

The Engine Abstraction

The starting point is a simple abstract base class that defines what any TTS engine needs to do:

from abc import ABC, abstractmethod
import numpy as np

class TTSEngine(ABC):
    @abstractmethod
    def generate(self, text: str) -> np.ndarray:
        ...

    @property
    @abstractmethod
    def sample_rate(self) -> int:
        ...

Two methods, that's it. generate takes a string of text and returns a numpy array of audio samples. sample_rate returns the sample rate in Hz so the caller knows how to save the audio correctly. Any TTS backend — Kokoro, Qwen, or something else entirely — just needs to implement these two things.

AudioGenerator then wraps any engine and handles the model-agnostic parts:

class AudioGenerator:
    def __init__(self, engine: TTSEngine) -> None:
        self._engine = engine

    def generate(self, text: str) -> np.ndarray:
        return self._engine.generate(text)

    def save(self, audio: np.ndarray, output_file: str) -> None:
        sf.write(output_file, audio, self._engine.sample_rate)

    def generate_and_save(self, text: str, output_file: str) -> None:
        self.save(self.generate(text), output_file)

The key thing here is save — it pulls sample_rate from the engine rather than hardcoding it. Different engines could theoretically produce audio at different sample rates, and this handles that transparently. generate_and_save is just a convenience method that chains the two together.

Loading the Model

The QwenCustomVoiceEngine constructor handles model loading. If you don't pass in a pre-loaded model, it figures everything out from the model_size parameter:

QWEN_MODEL_SIZES = {
    '0.6b': 'Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice',
    '1.7b': 'Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice',
}

def __init__(self,
             speaker: str = 'vivian',
             language: str = 'Auto',
             instruct: str | None = None,
             model_size: str = '0.6b',
             model: Qwen3TTSModel | None = None) -> None:
    if model is None:
        model_id = QWEN_MODEL_SIZES.get(
            model_size.lower(), QWEN_MODEL_SIZES['0.6b']
        )
        attn_impl = 'sdpa'
        try:
            import flash_attn
            attn_impl = 'flash_attention_2'
        except ImportError:
            pass
        device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
        model = Qwen3TTSModel.from_pretrained(
            model_id,
            device_map=device,
            dtype=torch.bfloat16,
            attn_implementation=attn_impl,
        )

The QWEN_MODEL_SIZES dictionary maps friendly size names to full Hugging Face model identifiers. This way the rest of the code just passes around "0.6b" or "1.7b" instead of long model strings. It also means that if Qwen releases new checkpoints, there's only one place to update. Note to self: I should really allow you to pass a full model name here and only use this dictionary for shorthands. I need to implement that still.

There are a few things worth noting about the from_pretrained call.

device_map controls where the model runs. The code checks torch.cuda.is_available() and uses the GPU if present, falling back to CPU otherwise. This is the same pattern that the Kokoro engine uses, so both engines behave consistently.

dtype=torch.bfloat16 halves the memory footprint compared to full float32 precision with negligible quality loss. For the 1.7B model, this is the difference between fitting on a consumer GPU and not fitting at all.

The attention implementation check is about GPU memory efficiency. FlashAttention 2 is an optimized attention algorithm that reduces VRAM usage during inference. But it requires the flash-attn package, which compiles from source and needs the CUDA Toolkit and a C++ compiler installed — a nontrivial setup on Windows. If it's not available, the engine falls back to PyTorch's built-in scaled dot product attention (sdpa), which works fine but uses a bit more VRAM. The code handles this gracefully: try the import, use it if it's there, move on if it's not. To be frank, I never got flash attention working — getting the CUDA Toolkit and C++ compiler set up on Windows was more than I wanted to take on right now. So the code is there for when I get around to it, but for now I use sdpa.

The constructor also accepts a pre-loaded model via the model parameter. This is useful for testing — you can inject a mock — and it also means you could share a single model instance across multiple engine objects if you needed to.

Generating Speech

The generate method is where text actually becomes audio:

def generate(self, text: str) -> np.ndarray:
    kwargs = {
        'text': text,
        'language': self._language,
        'speaker': self._speaker,
    }
    if self._instruct is not None:
        kwargs['instruct'] = self._instruct

    wavs, sr = self._model.generate_custom_voice(**kwargs)
    self._sample_rate = sr
    return wavs[0]

It assembles the keyword arguments for generate_custom_voice, conditionally including the instruct parameter, makes the call, and returns the first waveform.

The instruct parameter is only included when it's not None. This matters because the 0.6B model doesn't support instruction control — only the 1.7B CustomVoice model does. Passing instruct to the 0.6B model won't cause an error, but it will be silently ignored.

generate_custom_voice returns a list of waveforms because the Qwen3-TTS API supports batched generation — you can pass a list of strings and get back multiple audio arrays in one call. For our paragraph-at-a-time use case, we always pass a single string and take wavs[0]. However, I should really change things to allow this code to handle everything at once as an option. As I mentioned in the previous post, Qwen3-TTS produces a somewhat different voice each time you call it, which makes for a questionable audiobook experience when you're generating paragraph by paragraph. Batching everything into a single call might help with that consistency.

The sample rate is captured from the return value rather than hardcoded, though in practice Qwen3-TTS always returns 24000 Hz. By reading it from the response, the code stays correct even if a future model version changes the rate.

Wiring It Together

The CLI entry point in book_to_audio.py ties everything together. When the user passes --engine qwen, a small factory function creates the right engine:

def _create_engine(args):
    if args.engine == 'qwen':
        return QwenCustomVoiceEngine(
            speaker=args.speaker,
            language=args.language,
            instruct=args.instruct,
            model_size=args.model_size,
        )
    else:
        return KokoroEngine(voice=args.voice)

That engine gets wrapped in an AudioGenerator, which gets handed to BookToAudio, which does the actual document processing. BookToAudio doesn't know or care whether it's using Kokoro or Qwen — it just calls generate and gets audio back.

This is the payoff of the strategy pattern. Adding a third engine later — say, for voice cloning with the Qwen3-TTS Base model — means writing a new engine class, adding an option to _create_engine, and nothing else changes.

What's Next

Voice cloning is the natural next step. The Qwen3-TTS Base model can clone a voice from just a few seconds of reference audio, which opens up the possibility of generating an entire audiobook in a specific narrator's voice. That will be a separate engine class since it uses a different model and a different API (generate_voice_clone instead of generate_custom_voice), but the abstraction is already in place to support it.

If you need help with your Artificial Intelligence solutions, we're here to help.

Refactoring the Book2Audio Parsers

Thu, 30 Apr 2026 00:00:00 -0600

This is a progress update on Book2Audio — a tool that converts PDF and EPUB books into audio files using text-to-speech.

What We Did

Book2Audio has two parsers: DoclingParser for PDFs and EpubParser for EPUBs. Both do similar things — extract text from a document, clean it up, and chunk it into paragraphs — but they were implemented quite differently under the hood.

This update brings them into full alignment across several dimensions.

Shared Paragraph Accumulation

We extracted the shared paragraph accumulation logic into a new TextProcessor class. Both parsers now produce RawChunk objects and hand them off to TextProcessor, which handles the decisions about when to accumulate and when to emit a paragraph. Chunking behavior is now consistent across PDF and EPUB sources.

Unified Text Cleaning

Previously, PDF cleaning happened in DoclingParser and EPUB cleaning happened in EpubParser, with two separate cleaning functions that overlapped significantly. We merged these into a single clean_text function in general_utils.py and moved all cleaning into TextProcessor itself. Parsers now pass completely raw text — they extract and label, nothing more.

Cleaning now happens in a single upfront pass over all chunks before the accumulation loop runs. This gives the cleaner full context and sets up nicely for the LLM cleaning step described below.

We also took the opportunity to fix a long-standing bug in is_sentence_end — it wasn't recognizing curly quote characters as sentence-ending, which caused incorrect paragraph accumulation in some cases.

Consistent Parser Design

We also made EpubParser consistent with DoclingParser in several other ways:

Both now accept either a file path or a pre-loaded document object, making unit testing much simpler and eliminating the need for patch() in tests
Both are configured entirely at construction time — no configuration passed into run()
Both live in a parsers/ package and inherit from a shared BaseParser abstract base class
The CSV-based section skipping was removed from EpubParser — the caller loads the CSV and passes the result in, consistent with how DoclingParser handles page ranges

Why It Matters

The immediate benefit is cleaner, more testable code and consistent behavior between the two parsers. But the real motivation is what comes next: an LLM-based text cleaning step.

The plan is to add a DSPy module that operates on a sliding window of raw chunks before accumulation. It will:

Remove footnotes that slipped into paragraph text
Fix OCR errors and encoding artifacts that rule-based cleaning can't handle contextually
Join words and paragraphs that were incorrectly split across page breaks

The window is bounded by the same logic already used for accumulation — section headers are hard boundaries, so the LLM never sees text across a section break. This keeps the LLM's job well-scoped and the context meaningful.

Having both parsers produce output through the same TextProcessor pipeline means the LLM cleaner will work identically regardless of whether the source was a PDF or an EPUB. And for books available in both formats, the cleaner EPUB output can serve as reference data for training the LLM to clean up the noisier PDF version.

If you need help with your Artificial Intelligence solutions, we're here to help.

What Exactly Is an Inductive Bias?

Tue, 28 Apr 2026 00:00:00 -0600

In a previous post, I walked through Tom Mitchell's proof that a learner with no prior assumptions cannot generalize at all. The conjunctive restriction on our hypothesis space was doing all the real work -- without it, the algorithm was paralyzed. I promised a follow-up that would define that concept more precisely.

Mitchell calls it the learner's inductive bias. He defines this as:

"...the inductive bias of a learner as the set of additional assumptions B sufficient to justify its inductive inferences as deductive inferences." (Mitchell, p. 44)

Here is his formal definition:

The inductive bias of a learner L is any minimal set of assertions B such that for any target concept c and corresponding training examples D, (B + D + x) deductively entails L(x, D) for all new instances x. (Mitchell, p. 44)

Compare this to Wikipedia's definition of an 'inductive bias':

The inductive bias (also known as learning bias) of a learning algorithm is the set of assumptions that the learner uses to predict outputs of given inputs that it has not encountered (link)

In plain language: the inductive bias is whatever you'd have to add to the training data so that the learner's predictions follow by pure deduction. It is the gap between what the data says and what the learner concludes. It may be explicit, or implicit.

In the previous post, we saw that the Candidate-Elimination algorithm's inductive bias was the assertion "the target concept can be expressed as a conjunction of the attributes Genre, Mood, and Pacing." Feed that assertion to a deductive theorem prover along with the training data, and you get the same output as the so-called "inductive" learning algorithm. The "induction" was deduction plus an unstated assumption. This is why Popper was actually correct that there is no 'induction' per se. The so-called 'inductive' algorithms work just fine in real life -- but what is actually happening under the hood is always equivalent to deduction once the background knowledge is taken into consideration.

This definition is not limited to one algorithm. Every learner has an inductive bias, and identifying it tells you exactly what assumptions the learner is smuggling in.

Comparing Learners by Their Bias

As Mitchell puts it:

One advantage of viewing inductive inference systems in terms of their inductive bias is that it provides a nonprocedural means of characterizing their policy for generalizing beyond the observed data (Mitchell, p. 44)

To understand what he means, consider a "Rote-Learner" algorithm. All it does is store every training example in memory, and when you ask it to classify a new instance, it looks for an exact match. If it has seen that exact instance before, it returns the stored classification. Otherwise, it refuses to classify it at all. This algorithm has no inductive bias -- and it is misleading to even call it a "learner" because it never actually learns anything. It just stores data points. It is the "unbiased learner" from the previous post, and as we saw there, it is completely useless for generalization. (Mitchell, p. 45)

Now compare this to the Candidate-Elimination algorithm from the previous post. It classifies a new instance only when every hypothesis in the version space agrees on the answer. Its inductive bias is a single assumption: the target concept is contained in the hypothesis space H, which is the space of conjunctions between attributes. (i.e. conjunction means the attributes can be "AND"ed but not "OR"ed together.) Because it has a stronger bias than the Rote-Learner, it can classify instances the Rote-Learner cannot. But the correctness of those classifications depends entirely on whether the bias is true -- whether the target concept really is in H. If it is not, the version space may collapse entirely -- or worse, the algorithm may converge on the wrong answer. (Mitchell, p. 45).

Mitchell also describes the Find-S algorithm, which only tracks the S boundary -- the most specific hypothesis consistent with the positive examples. It ignores negative examples entirely and uses that single hypothesis to classify everything: if the hypothesis covers a new instance, it predicts positive; otherwise, it predicts negative. This gives it an even stronger bias than the Candidate-Elimination algorithm. In addition to assuming the target concept is in H, it assumes that all instances are negative unless the opposite is entailed by its other knowledge (Mitchell, p. 45). Where Candidate-Elimination would say "I don't know" when the version space is split, Find-S always has an answer: negative. The advantage is that it now always gives an answer, unlike the Candidate-Elimination algorithm. But this comes at the cost of sometimes being confidently wrong. This makes it the most aggressive of the three -- and the most dependent on its assumptions being correct.

The pattern is clear: stronger bias means more generalization, but also more risk. As Mitchell notes, "more strongly biased methods make more inductive leaps, classifying a greater proportion of unseen instances" (Mitchell, p. 45). But those leaps are only as good as the assumptions that enable them.

The pattern here generalizes. All learning algorithms can be characterized in terms of their inductive bias. Some biases are categorical restrictions that completely rule out certain concepts -- like the assumption that H contains the target concept (e.g. Candidate-Elimination algorithm). Others merely express preferences, ranking some hypotheses above others -- like "prefer simpler hypotheses over complex ones." And some inductive biases are hardwired into the algorithm's design. But it is even possible to create a learner that can modify its own inductive bias. (Mitchell, p. 45).

Restriction Bias vs. Preference Bias

Not all biases work the same way. Mitchell draws an important distinction between two kinds (Mitchell, pp. 62-64).

A restriction bias limits which hypotheses the learner can consider at all. The Candidate-Elimination algorithm has a restriction bias -- it literally cannot represent hypotheses outside its hypothesis space. If the true concept is a disjunction (i.e. "OR"s) and the space only allows conjunctions ("AND"s), the algorithm will never find it. The advantage is that within its restricted space, it searches completely -- it considers every consistent hypothesis (Mitchell, p. 64).

A preference bias does not restrict the hypothesis space but instead orders it -- the learner prefers some hypotheses over others. Decision tree learners like ID3 have a preference bias. The hypothesis space of decision trees can represent any discrete-valued function -- it is univeral! But ID3 does not search that space exhaustively. It uses a greedy, top-down strategy that favors shorter trees and places the most informative attributes near the root (Mitchell, p. 62). Its inductive bias is entirely a consequence of this search strategy, not the expressiveness of its representation.

A great example of a preference bias is genetic programming. Its hypothesis space is the set of all programs that can be composed from a given set of primitives. Since nature is computable (see the Church-Turing-Deutsch thesis), this means its search space is in principle universal -- it can represent anything. There is no restriction bias at all. But it still has an inductive bias. As Mitchell notes, the performance of genetic programming depends crucially on the choice of representation and on the choice of fitness function (Mitchell, p. 266). And realistically, the space of all programs is so vast that the algorithm will only ever explore a tiny fraction of it -- mostly relatively small programs. The evolutionary search strategy -- selection, crossover, mutation -- determines which programs get explored and which get discarded. The bias is entirely in how it searches, not in what it can represent.

Mitchell is direct about which is generally better: a preference bias is typically more desirable than a restriction bias, because it allows the learner to work within a complete hypothesis space that is guaranteed to contain the target function. A restriction bias always carries the risk that you have excluded the right answer from the start (Mitchell, p. 64).

Neural Networks: A Different Kind of Bias

When Mitchell turns to neural networks trained with backpropagation, the inductive bias becomes harder to pin down -- but it does not disappear (Mitchell, pp. 104-107).

The hypothesis space is now continuous rather than discrete. As Mitchell describes it, "every possible assignment of network weights represents a syntactically distinct hypothesis that in principle can be considered by the learner" -- the hypothesis space is the n-dimensional Euclidean space of all the network's weights (Mitchell, p. 106). And the search strategy is completely different from the algorithms we have been discussing. Backpropagation is essentially a hill-climbing algorithm that uses calculus to determine which direction is downhill. It follows the slope of the error surface, taking small steps in whatever direction reduces the error most. This means it can get trapped in local minima -- valleys that are not the deepest valley -- and is only guaranteed to converge toward some local minimum, not necessarily the global one (Mitchell, p. 104).

So what is the inductive bias? Mitchell is candid that "it is difficult to characterize precisely the inductive bias of BACKPROPAGATION learning, because it depends on the interplay between the gradient descent search and the way in which the weight space spans the space of representable functions." But he offers an approximate answer: "one can roughly characterize it as smooth interpolation between data points." Given two positive training examples with no negative examples between them, backpropagation will tend to label points in between as positive as well (Mitchell, pp. 106-107).

This is still an inductive bias -- an assumption that goes beyond what the data logically entails. The data does not say anything about what lies between the training examples. The network's architecture and training procedure are assuming the answer is smooth. That assumption is what makes generalization possible, and it is also what makes generalization fallible.

Why This Matters

Every time a learning algorithm says anything about an instance it has not seen, it is going beyond what the data alone can justify. It is making an assumption. The inductive bias is that assumption, named and made explicit.

This is Popper's point. You cannot get from observations to general theories without bringing something to the table. The question is never whether you have prior assumptions -- you always do. The question is whether they are good ones.

An Optimal Inductive Bias?

This does raise an interesting question. Is there an "optimal" inductive bias? One whose search space is universal -- every possible function -- but whose search strategy is still efficient and tractable for any problem? We have seen that genetic programming and decision trees are both universal in their hypothesis space -- they can in principle represent any function. But universality alone is not enough. Both still depend on their search strategy to find the right hypothesis, and that search strategy is where the bias lives. And we have seen that stronger biases enable more generalization but risk being wrong. Is there a sweet spot? That is a question for a future post.

All page references to Mitchell are from Machine Learning (McGraw-Hill, 1997).

If you need help with your Artificial Intelligence solutions, we're here to help.

Adding EPUB Support to Book2Audio

Mon, 20 Apr 2026 00:00:00 -0600

Book2Audio started as a PDF-to-audiobook converter, but a lot of the best books come as EPUBs. This post covers how we added EPUB support by migrating code from our earlier BookSearchArchive project — a RAG-based book search tool discussed in a previous post.

The code referenced in this post can be found at this specific commit in the Book2Audio repository.

Where the Code Came From

BookSearchArchive included an EPubLoader Haystack component that loaded EPUB files and returned raw HTML sections, and an HTMLParser class that parsed those HTML sections into text chunks suitable for embedding and semantic search. Neither was designed for audio — they were optimised for RAG pipelines, returned Haystack ByteStream objects, and included features like double_notes that made sense for search but not for listening.

The goal was to strip out the Haystack dependency, clean up the interface, and make EPUB parsing a drop-in replacement for PDF parsing in Book2Audio.

What Changed

New File: epub_parser.py

This is the heart of the EPUB support. The EpubParser class takes a path to an .epub file and produces the same output as DoclingParser — a list of cleaned paragraph strings and a list of metadata dicts.

Single file, consistent interface. Rather than a loader/parser split like BookSearchArchive, EpubParser handles everything in one class:

parser = EpubParser("my_book.epub", meta_data={}, min_paragraph_size=300)
docs, meta = parser.run()

HTML parsing via BeautifulSoup. EPUBs are zipped HTML files. We use ebooklib to read the EPUB and extract each section's HTML, then BeautifulSoup to traverse the tag tree. The recursive_yield_tags function walks the HTML and yields leaf tags containing text, skipping structural elements like divs.

Chapter and section titles are emitted as paragraphs. In the RAG version, titles were stored only in metadata. For audio they need to be read aloud, so chapter titles and section headers are emitted as their own standalone paragraphs before the section content.

Footnote removal. The remove_footnotes parameter strips superscript tags from paragraphs unless they appear as the first content — which usually means they are footnote markers at the start of a footnote paragraph rather than inline citations.

Sections to skip. Two mechanisms are supported: a sections_to_skip.csv file in the same directory as the EPUB, and a sections_to_skip parameter passed directly to run(). Both are additive.

Debug output. Calling run(generate_text_file=True) writes two files alongside the EPUB:

<n>_processed_paragraphs.txt — the cleaned paragraph text
<n>_processed_meta.txt — metadata alongside each paragraph, useful for verifying chapter and section attribution

Updated: general_utils.py

The refactor revealed that a lot of text cleaning logic was duplicated or misplaced. We moved reusable utilities from docling_utils.py into general_utils.py:

is_sentence_end and is_ends_with_punctuation — pure string functions with no DocItem dependency
is_roman_numeral and enhance_title — migrated from parse_utils.py in BookSearchArchive
load_sections_to_skip — CSV loading logic, shared between EPUB and potentially other formats
The full clean_text pipeline — whitespace, hyphens, quotes, punctuation spacing, bracket spacing, apostrophes

docling_utils.py now focuses on what it should: DocItem inspection helpers and clean_pdf_text, which extends clean_text with PDF-specific steps for ligature normalisation, encoding artifact correction, and footnote number stripping.

Updated: book_converter.py

convert_to_audio now dispatches by file extension:

if suffix == '.pdf':
    # DoclingParser
elif suffix == '.epub':
    # EpubParser
elif suffix == '.txt':
    # read and convert directly

The sections_to_skip parameter threads all the way from the command line through main() and convert_to_audio to EpubParser.run().

Using It

From the Command Line

Convert an EPUB to audio:

python book_to_audio.py "documents/my_book.epub"

Dry run with debug output to inspect what the parser extracted:

python book_to_audio.py "documents/my_book.epub" --dry-run --generate-text-file

Skip front matter and navigation sections:

python book_to_audio.py "documents/my_book.epub" --sections-to-skip cover titlepage toc

Command Line Parameters

file_path — path to the EPUB file
--dry-run — parse the document but skip audio generation
--generate-text-file — save processed paragraph and metadata files alongside the source EPUB
--sections-to-skip — one or more section IDs to skip, separated by spaces
--engine — TTS engine to use: kokoro (default) or qwen
--voice — Kokoro voice identifier (default: af_heart)
--speaker — Qwen speaker name (default: vivian)
--output-file — path to the output WAV file

From Python

from epub_parser import EpubParser

parser = EpubParser(
    source="documents/my_book.epub",
    meta_data={"source": "my_book.epub"},
    min_paragraph_size=300,
    remove_footnotes=True
)
docs, meta = parser.run(
    generate_text_file=True,
    sections_to_skip=["cover", "titlepage", "toc"]
)

The --generate-text-file flag is especially useful for EPUBs where section IDs vary between publishers and you need to identify which ones to skip. Run with --dry-run --generate-text-file first, inspect the output, then add unwanted section IDs to --sections-to-skip.

What We Left Behind

A few things from BookSearchArchive did not make the cut for this version:

double_notes — in the RAG version, sections titled "Notes" got double the minimum paragraph size to avoid dominating search results. This does not apply to audio.
min_section_size — the RAG pipeline skipped sections below a minimum total length. For audio we want everything.
The Haystack component wrapper — EPubLoader and HTMLParserComponent were Haystack-specific. All of that is gone.
Multiple file paths — EPubLoader accepted a list of files. EpubParser takes one file, consistent with DoclingParser.

Files Added or Modified

epub_parser.py — new
utils/general_utils.py — significantly expanded
utils/docling_utils.py — slimmed down, PDF-specific logic only
book_converter.py — EPUB dispatch in convert_to_audio
book_to_audio.py — sections_to_skip parameter in main()
tests/test_epub_parser.py — new unit tests
tests/test_general_utils.py — new unit tests for moved utilities
tests/test_document_output.py — extended to cover EPUB integration tests

If you need help with your Artificial Intelligence solutions, we're here to help.

Implementing Qwen3-TTS in My PDF-to-Audiobook Pipeline (Qwen3-TTS, Part 1)

Fri, 17 Apr 2026 10:32:51 -0600

In my last post, I walked through building a PDF-to-audiobook pipeline using Kokoro for text-to-speech. The pipeline worked well enough that I've been actively using it to listen to books that only exist as PDFs. But I mentioned wanting to try Alibaba's recently open-sourced Qwen3-TTS as an alternative voice engine, and I've now done exactly that. (My code is found in my github repo.)

This post covers how to use Qwen3-TTS to generate speech from the command line and a discussion of how I refactored the code to support multiple TTS engines. A follow-up post will walk through the Qwen engine code itself.

Trying Qwen3-TTS

Before touching any of my existing code, I wanted to hear what Qwen3-TTS actually sounded like. The setup is straightforward. Install the package:

pip install -U qwen-tts

The first time you use a model, the weights download automatically from Hugging Face. The 0.6B model is roughly 1.2GB and the 1.7B model is around 3.4GB.

Once installed, generating speech from Python is only a few lines:

import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice",
    device_map="cuda:0",
    dtype=torch.bfloat16,
)

wavs, sr = model.generate_custom_voice(
    text="The philosopher argued that all knowledge is provisional.",
    language="English",
    speaker="ryan",
)

sf.write("output.wav", wavs[0], sr)

Code found here.

You load a model with from_pretrained, call generate_custom_voice with your text, a language, and a speaker name, and you get back a list of waveform arrays and a sample rate. Write the first waveform to a file and you have a WAV you can play.

Qwen3-TTS comes with nine built-in speakers: aiden, dylan, eric, onoanna, ryan, serena, sohee, unclefu, and vivian. They vary quite a bit in tone and accent. I'd recommend generating a short sample with each one to find what works for your use case. However, my experience is that you get a somewhat different voice each time you run the voice. This makes it less than desirable for reading an audio book.

The 1.7B CustomVoice model also supports an instruct parameter that lets you control the delivery style with natural language:

wavs, sr = model.generate_custom_voice(
    text="The philosopher argued that all knowledge is provisional.",
    language="English",
    speaker="ryan",
    instruct="Read in a calm, steady audiobook narration style",
)

This is a genuinely interesting feature, but be aware that instruction control only works on the 1.7B models. The 0.6B models silently ignore the instruct parameter.

Find a list of all the available models on Hugging Face here.

Refactoring for Multiple Engines

My original code for Book2Audio had the Kokoro TTS model wired directly into the AudioGenerator class. To support Qwen3-TTS as an alternative, I needed to pull the model-specific logic out and make it swappable. (Apologies for naming the repo Book2Audio and the python file booktoaudio. I need to rename the repo at some point to match.)

The approach was a straightforward application of the strategy pattern. I created a TTSEngine abstract base class with two methods: generate, which takes text and returns a numpy audio array, and a sample_rate property. Then I wrote two concrete implementations: KokoroEngine wrapping the existing Kokoro pipeline, and QwenCustomVoiceEngine wrapping Qwen3-TTS.

AudioGenerator, which previously owned the Kokoro pipeline directly, now takes any TTSEngine. It delegates audio generation to whatever engine it's given and handles only the model-agnostic work of saving WAV files. BookToAudio, the class that orchestrates document parsing and paragraph-by-paragraph generation, didn't need to change at all. It still talks to AudioGenerator the same way it always did.

I also split the single book_to_audio.py file into several files. The engines live in their own directory, AudioGenerator and BookToAudio each got their own module, and book_to_audio.py became a thin CLI entry point. This makes it easy to add more engines later without everything piling up in one file.

From the command line, switching engines is just a flag:

python book_to_audio.py "documents/MyBook.pdf" --engine kokoro --voice af_heart

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --language English

You can also convert plain text directly without a PDF:

python book_to_audio.py --text "Hello world" --engine qwen --speaker vivian

By default, the Qwen engine uses the 0.6B model. To use the larger 1.7B model, which supports instruction control:

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --language English --model-size 1.7b --instruct "Read in a calm, steady audiobook narration style"

For long documents, you can limit the page range to test on a small section before committing to a full run:

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --start-page 10 --end-page 15

To process the document without generating audio — useful for inspecting the extracted text before spending time on generation:

python book_to_audio.py "documents/MyBook.pdf" --dry-run --generate-text-file

And to specify an output file name instead of the default:

python book_to_audio.py "documents/MyBook.pdf" --engine qwen --speaker ryan --output-file "my_audiobook.wav"

The Kokoro path works exactly as before. The Qwen path adds a few extra options for speaker, language, model size, and style instructions.

Kokoro vs. Qwen3-TTS for Audiobooks

After testing both engines on the same material, I have to be honest: I still prefer Kokoro for most audiobook listening. The Qwen3-TTS voices, while technically impressive, tend to have cadence patterns and occasional accent shifts that can be fatiguing over long listening sessions. The ryan speaker comes closest to a natural audiobook narrator in English, and I may switch to it in the future as I experiment more with the instruct parameter on the 1.7B model. But for now, Kokoro's more neutral delivery wins for extended listening.

Hardware Considerations

One thing that surprised me during this process was discovering that my laptop had been running Kokoro on CPU the whole time. PyTorch had been installed without CUDA support, which meant torch.cuda.is_available() returned False and everything silently fell through to CPU inference. It worked, just slower than it needed to be.

If you're running this on a machine with an NVIDIA GPU, make sure you install the CUDA-enabled version of PyTorch:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

You can verify it worked with:

python -c "import torch; print(torch.cuda.is_available())"

In any case, my code will use your GPU if it's available and CUDA is properly installed.

For Qwen3-TTS specifically, the 0.6B model needs roughly 1.5GB of VRAM and runs comfortably on a 6GB laptop GPU. The 1.7B model needs 4-6GB and may be tight on consumer hardware, especially if other applications are using the GPU. I'd recommend starting with the 0.6B model and only moving to the 1.7B if you want instruction control or find the quality insufficient. Theoretically the 1.7B should work, but I haven't really tested it yet on my laptop. I'll do that in a future post.

What's Next

In the next post, I'll walk through the actual Qwen engine code, explaining how it works and how to use the Qwen3-TTS API. I also plan to explore Qwen3-TTS voice cloning, which lets you train the model on a specific narrator's voice from just a short audio clip and then generate an entire audiobook in that style.

If you need help with your Artificial Intelligence solutions, we're here to help.

Machine Learning 101: The Key Concepts Behind Every Learning Algorithm

Fri, 10 Apr 2026 10:53:35 -0600

Machine learning textbooks have their own vocabulary. But behind the jargon lies a process that would be deeply familiar to Karl Popper: conjecture and refutation. This post is a short reference guide to the key terms from Tom Mitchell's Machine Learning -- a foundational textbook that I also draw on in my post on the futility of unbiased learning. For each term, I will give Mitchell's definition and then show how it maps onto Karl Popper's logic of scientific discovery.

To make this concrete, imagine a 19th-century physician trying to figure out what causes a mysterious illness sweeping through a city. Patients come in with various combinations of age, diet, water source, neighborhood, and occupation. Some get sick, others do not. The physician is trying to discover the underlying rule from these observations.

Here is the truth the physician does not yet know: the illness strikes patients who drink from the river and live in the low-lying district near the tannery due to contamination from the tannery. Both conditions must be present -- river drinkers in the hills stay healthy, and tannery district residents who drink from wells stay healthy. Only the combination is deadly.

This is, as Mitchell would put it, a concept learning task (Mitchell, p. 22). The physician's job is to discover that two-attribute rule from a handful of patients -- without knowing in advance how many attributes matter or which ones.

Instance Space

In machine learning, the instance space (denoted X) is the set of all possible examples the learner could encounter (Mitchell, p. 22). In our medical example, it is the set of all possible patients -- every combination of age, diet, water source, neighborhood, occupation, and so on. A young well-drinking hillside baker. An old river-drinking tannery district laborer. Every combination, whether or not the physician has actually seen such a patient.

In Popper's framework, the instance space is the set of all possible observations or experiments. It defines the scope of what the theory is about. Most of these possible observations will never actually be made. But they all matter, because a good theory must make predictions about all of them -- not just the ones we happen to have seen.

Target Concept

The target concept (denoted c) is the true rule the learner is trying to discover (Mitchell, p. 22). It is a function that correctly classifies every instance. In our example, the target concept is: a patient gets sick if and only if they drink from the river AND live in the tannery district. The target concept is unknown to the physician. The whole point of learning is to figure out what it is.

In Popper's terms, the target concept is the law of nature we are searching for. We do not have direct access to it. We can only approach it indirectly through conjecture and refutation -- proposing theories and testing them against observations. We may never know for certain that we have found it, but we can know when we have not found it, because our conjecture will be refuted by the evidence.

Hypothesis

A hypothesis (denoted h) is a candidate theory -- one possible answer to the question "what is the target concept?" (Mitchell, p. 23). The physician might conjecture "patients who drink from the river get sick" or "patients who live in the tannery district get sick" or "old patients get sick." Each of these is a hypothesis. Some are too broad, some are too narrow, and one -- "river drinkers in the tannery district get sick" -- happens to be correct. But the physician does not know that yet.

A hypothesis is exactly what Popper calls a conjecture. It is a bold guess about the structure of reality. It may be right or wrong. What matters is that it is testable -- it makes predictions that can be checked against observations.

Hypothesis Space

The hypothesis space (denoted H) is the set of all hypotheses the learner is willing to consider (Mitchell, p. 23). This is not the set of all possible explanations -- it is the set of explanations the learner can express given its representation.

It is intractable to consider every possible hypothesis, as this would be an infinite set. But this is unnecessary. We have a lot of background knowledge that lets us constrain the possible hypotheses we will consider. So typically we start with an already partially constrained hypothesis space based on our background knowledge. This is part of our own human "inductive bias."

But suppose our physician only considers single-attribute hypotheses -- "it is the water source" or "it is the neighborhood" -- then the correct two-attribute answer is not even in the hypothesis space. The physician could examine every patient in the city and still never find the answer, because the target concept cannot be expressed within the hypothesis space he is considering. The choice of hypothesis space determines what the learner can and cannot discover.

In Popper's framework, the hypothesis space corresponds to the theoretical framework within which the scientist operates. No scientist considers every conceivable theory. They work within a tradition, a paradigm, or a set of background assumptions that constrain which conjectures are even formulable. As I discussed in my post on [inductive bias][2], this constraint is not a flaw -- it is a prerequisite for learning anything at all. But it must be the right constraint, or the truth will be invisible to you.

Training Examples

Training examples (denoted D) are the actual observations available to the learner (Mitchell, p. 23). Each training example is an instance paired with its correct classification. A positive example is a patient who got sick. A negative example is a patient who stayed healthy.

Suppose the physician has seen five patients so far:

Old, river, tannery district, laborer -- sick
Young, river, tannery district, baker -- sick
Old, river, hillside, farmer -- healthy
Young, well, tannery district, tanner -- healthy
Old, well, hillside, laborer -- healthy

In Popper's framework, these are the observations against which we test our conjectures. The third patient -- a river drinker who stayed healthy -- is a falsification of the hypothesis "river water causes the illness." The fourth patient -- a tannery district resident who stayed healthy -- falsifies "living in the tannery district causes the illness." And the fifth patient -- an old laborer who stayed healthy -- falsifies "being a laborer causes the illness." Only the conjunction "river AND tannery district" survives all five observations.

As Popper argued, falsifications are where the real action is. A thousand sick river-drinking tannery residents do not prove that the combination is the cause -- they are merely consistent with it. But each negative example eliminates entire families of wrong theories in one stroke.

Concept Learning

Mitchell defines concept learning as "inferring a boolean-valued function from training examples of its input and output" (Mitchell, p. 21). That sounds narrow, but it is surprisingly general. Our physician is doing concept learning: he has patients (instances), he knows which ones got sick and which did not (positive and negative examples), and he is searching for the rule that separates them. But this is also what scientists do all the time. Any time we seek a causal explanation -- what causes this disease, what makes this material brittle, why do some stars explode and others do not -- we are doing concept learning. We have observations, we have outcomes, and we are searching for the rule.

But here is the key reframing that Popper would appreciate: Mitchell treats concept learning as a search problem. The learner is searching through a hypothesis space for a hypothesis consistent with the training data (Mitchell, p. 23). Our physician is not staring at patients and waiting for a pattern to emerge. He is starting with a space of possible theories and eliminating the ones that fail. The hypothesis "it is the water" fails when he sees a healthy river drinker in the hills. The hypothesis "it is the neighborhood" fails when he sees a healthy well drinker in the tannery district. What remains is not what the data induced but what the data failed to refute. This is exactly what Popper and Donald Campbell called "evolutionary epistemology" -- knowledge grows not by accumulation but by variation and selection. You generate candidate theories, test them against reality, and the ones that survive criticism are what remain. The data does not build the theory. The data selects among the theories.

Version Space

The version space is the set of all hypotheses in H that are consistent with the training data observed so far (Mitchell, p. 31). As training examples come in, hypotheses that are contradicted by the data get eliminated and the version space shrinks.

After the physician's first two patients (both sick, both river-drinking tannery district residents), many hypotheses are still alive: maybe it is the river, maybe it is the district, maybe it is the combination, maybe it is being a laborer. The version space is large. But after the third patient -- a healthy river drinker in the hills -- every hypothesis that blames the river alone is eliminated. After the fourth -- a healthy well drinker in the tannery district -- every hypothesis that blames the district alone is eliminated. The version space is closing in on the truth.

In Popper's terms, the version space is the set of conjectures that have survived all attempts at refutation so far. Each new observation either leaves it unchanged or shrinks it. The goal is to shrink it until only one hypothesis remains.

Consistent Hypothesis

A consistent hypothesis is one that correctly classifies every training example seen so far (Mitchell, p. 23). It has not been falsified by any observation.

After all five patients, the hypothesis "river AND tannery district" is consistent. But so are other, more specific hypotheses. Consider: we have not yet seen a river-drinking tannery district farmer. So the hypothesis "river AND tannery district AND NOT farmer" is equally consistent with everything we have observed -- maybe farmers are somehow immune. Likewise, both sick patients happened to be either laborers or bakers, so "river AND tannery district AND (laborer OR baker)" also fits the data. We simply have not seen enough patients to distinguish these hypotheses from each other.

So "consistent" does not mean "correct." It means "not yet refuted." This is precisely Popper's point about corroboration. A theory that has survived testing is corroborated, not confirmed. It has proven its mettle so far, but it remains permanently open to future refutation. The physician would need to find a river-drinking tannery district farmer to tell these hypotheses apart.

The General-to-Specific Ordering

Mitchell observes that hypotheses can be naturally ordered from general to specific (Mitchell, p. 24). A more general hypothesis classifies more instances as positive. The hypothesis "anyone who drinks from the river gets sick" is more general than "river drinkers in the tannery district get sick" -- the first covers a superset of the cases covered by the second.

This ordering has a direct Popperian interpretation. Popper argued that more general theories are more falsifiable -- they make bolder claims about the world, and therefore there are more ways they could be wrong. "All river drinkers get sick" is easier to refute than "river drinkers in the tannery district get sick," because the first makes predictions about a far wider range of patients. Popper considered this a virtue. Bolder theories, when they survive testing, tell us more about the world.

The Candidate-Elimination algorithm exploits this ordering to efficiently search the hypothesis space -- tracking the most general and most specific surviving hypotheses and using each new observation to tighten the bounds. As I described in my post on the futility of unbiased learning, this is falsification implemented as a computer program.

The Takeaway

Machine learning is not induction. It is search -- a search through a space of conjectures, guided by observations, eliminating the hypotheses that fail. The vocabulary is different, but the logic is the same logic Karl Popper described: bold conjectures, tested against experience, with the wrong ones ruthlessly discarded.

The key terms from Mitchell map cleanly onto this process:

The instance space is the domain of possible observations
The target concept is the unknown law we seek
A hypothesis is a conjecture
The hypothesis space is the set of conjectures we are willing to consider
Training examples are the observations that test our conjectures
Concept learning is the search for surviving conjectures
The version space is the set of conjectures not yet refuted
A consistent hypothesis is a conjecture that has survived all tests so far
The general-to-specific ordering reflects Popper's insight that bolder theories are more falsifiable

All page references to Mitchell are from Machine Learning (McGraw-Hill, 1997).

If you need help with your Artificial Intelligence solutions, we're here to help.

Book2Audio: Reviving My PDF-to-Audiobook Project (and Fighting Dependency Hell Along the Way)

Fri, 03 Apr 2026 12:38:46 -0600

A while back, I wrote about using Kokoro to convert a PDF into an audiobook. The core idea was straightforward: take a PDF, strip away the noise—headers, footers, page numbers, figure captions—and feed clean text into a text-to-speech model so the result actually sounds like someone reading a book, not someone reading a formatted document aloud.

To get there, I leaned on IBM's Docling for PDF-to-Markdown conversion, which does an impressive job of understanding page layout, reading order, and document structure. I also used NLTK to clean up hyphenated words that PDF extraction loves to leave behind—the kind where "understand-\ning" gets split across a line break and ends up in your audio as two separate words.

Picking It Back Up

Unfortunately, I never got back to that project. The original code was embedded inside my "Book Search Archive" app, which had grown large and unwieldy enough that even small changes felt like a chore. So I decided to start fresh with a dedicated repository focused entirely on the PDF-to-audio pipeline: Book2Audio on GitHub.

Clean slate, single purpose, no distractions. Simple, right?

The Docling Regression

It was not simple.

The first thing I discovered was that the latest version of IBM Docling simply could not read my test PDF past page 17. Every time it hit that page, the conversion would die with an error like this:

Stage preprocess failed for run 1, pages [17]: std::bad_alloc

That's a memory allocation failure deep inside the C++ PDF parsing backend. It appears to be a known but unfixed error. After a fair amount of troubleshooting—trying different configurations, different PDFs, different environments—I came to accept that the newer version of Docling just didn't handle my test document as well as the original version I'd used months ago. Software updates aren't always upgrades, especially when the underlying parser has been rearchitected.

Python Dependency Hell

Rather than keep fighting the latest release, I decided to roll back to the older version that had worked before. This turned out to be its own adventure. Pinning docling==2.14.0 is one thing; getting it to coexist peacefully with Kokoro, NLTK, and all of their transitive dependencies is another.

The pip resolver tried its best, but ultimately I had to hand-tune the versions myself. After a lot of trial and error, here's the incantation that finally worked:

bash pip install docling==2.14.0 kokoro==0.7.15 misaki==0.7.15 typer==0.12.5 numpy==1.26.4 opencv-python-headless==4.10.0.84 pip install nltk==3.9.1 pip install soundfile==0.13.1

There's a requirements.txt in the repo as well, but I'll be honest—your mileage may vary. This is one of the perennial frustrations of the Python ecosystem. The pip installer does a reasonable job checking dependencies in isolation, but it often can't resolve conflicts across packages that each have their own strong opinions about which version of NumPy or OpenCV they need. And some of these packages really want Python 3.12 or lower, which adds yet another variable.

If you've spent any time in Python-land, none of this will surprise you. But it's worth documenting, if only so future-me doesn't have to rediscover it.

But it Works!

If you follow the above instructions, it will work! I am actively using even this primitive version to convert PDFs into audio books. It's surprising how well it does given how unsophisticated it is.

Go to my github repo and clone it and then run it like this:

python book_to_audio.py "BookTitle.pdf"

There is a full readme file that explains the full details of how to use it. This is a decently working app even in this early state. And the code base has full unit tests.

What's Next

The current state of the repo is a working starting point: it can take a PDF, parse it with Docling, clean the text, and generate audio with Kokoro. But I have bigger plans.

Upgrading the voice with Qwen3-TTS. Kokoro is capable, but I want to try Alibaba's recently open-sourced Qwen3-TTS. What makes it especially interesting for audiobook generation is its contextual understanding—it can adapt tone, pacing, and emphasis based on the meaning of the text, not just its phonemes. It supports 10 languages, voice cloning from just 3 seconds of audio, and even voice design from natural language descriptions ("a warm, measured narrator voice"). For long-form audio like audiobooks, that kind of semantic awareness could make a real difference in listenability.

Smarter text cleanup with local LLMs. Right now, NLTK handles the text smoothing—fixing hyphenation, removing artifacts. But I want to experiment with running a local model through Ollama to do more intelligent cleanup: identifying and removing headers, footers, page numbers, figure references, and other extraneous text that shouldn't appear in an audiobook. An LLM can understand context in a way that regex and rule-based approaches can't. Maybe the ideal approach is a combination of both—NLTK for the mechanical stuff, and a small fine-tuned model for the judgment calls. This might even be a good candidate for fine-tuning a smaller model specifically for the task of stripping non-narrative content from documents.

The Book2Audio repo is public if you want to follow along or contribute. It's early days, but the foundation is in place.

If you need help with your Artificial Intelligence solutions, we're here to help.

Induction is a Myth: The Futility of Unbiased Learning

Thu, 12 Mar 2026 16:55:14 -0600

Karl Popper's Disproof of Induction

Karl Popper argued that it is logically impossible to derive a general theory from specific observations. You can stare at a million data points and no universal law will ever logically follow from them. The observations are always concrete and specific; the theory is always abstract and universal. You simply cannot get from one to the other by logic alone. (See Conjectures and Refutations: The Growth of Scientific Knowledge, pp. 251-253)

Thus induction is a myth. No "inductive logic" exists. And although there exists a "logical" interpretation of the probability calculus, there is no good reason to assume that this "generalized logic" (as it may be called) is a system of "inductive logic" (Unended Quest, p. 171)

Despite such a startlingly powerful disproof, most people assumed it must be wrong. They sought after a new kind of "inductive logic" that would allow you to somehow start with only observations and generalize to a universal law. The reasoning went that there must be such an inductive logic because we do—in practice—induce general laws all the time in science. Probability theory—particularly the Bayesian interpretation—was often advanced as the missing inductive logic that would bridge the gap. When it was discovered that probability theory could be thought of as "extending deductive logic" (by which they really meant extending propositional logic, not first order logic), many became convinced they had found what they were looking for. The arrival of Cox's theorem was often interpreted as cementing this idea.

And on top of all that, wasn't it just a fact that machine learning algorithms generalize from data every day? Your streaming service watches you binge a few movies and then somehow knows what to recommend next. That's induction, right? Isn't the machine deriving a general rule from specific observations? Popper must be wrong!

Tom Mitchell's "Futility of Bias-Free Learning"

Tom Mitchell—one of the foundational figures in machine learning—has a devastating answer to this question. In his textbook Machine Learning (McGraw-Hill, 1997), he proves something that should be far better known: a learner that makes no prior assumptions about what it's looking for cannot generalize at all. Not even a little. It can memorize what it has seen, but the moment you show it something new, it has literally no rational basis for making a classification. Mitchell calls this "The Futility of Bias-Free Learning" (Mitchell, p. 42).

What makes this so interesting is that Mitchell arrives at essentially the same conclusion as Popper, but from the completely opposite direction. Popper was a philosopher arguing against inductivism. Mitchell is a computer scientist trying to make induction work. And yet they converge on the same point: you cannot generalize from observations alone. You always need something else—some set of prior assumptions—to bridge the gap.

But Mitchell goes one step further. He shows that when you do add those prior assumptions, the resulting "inductive" algorithm is actually equivalent to a deductive theorem prover. The so-called induction was deduction all along, just as Popper claimed. It just didn't look like it.

Let me walk through how this works.

A Movie Recommendation Problem

Imagine a movie streaming service trying to figure out whether you'll enjoy a given film. To keep things tractable, suppose the system describes movies using three attributes:

Genre: Action, Comedy, Drama, Sci-Fi
Mood: Light, Dark
Pacing: Fast, Slow

Each movie is some combination of these attributes, and your reaction is binary: you either enjoyed it or you didn't. The service's job is to figure out the general rule—the target concept—that explains your taste.

This isn't meant to be a realistic example, but suppose you subconsciously only enjoy fast-paced movies with a dark mood, regardless of genre. The service is trying to figure that out and then recommend other such movies to you.

Now, there are 4 x 2 x 2 = 16 possible movie descriptions in this space. And the streaming service has watched you react to five or six of them. From those data points, it needs to learn a rule that correctly predicts your reaction to all the other movies you haven't seen yet.

This is, at its core, what Mitchell calls a "concept learning" task: searching through a space of possible hypotheses for the one that fits the observed data (Mitchell, p. 23).

How Candidate-Elimination Works (a.k.a. Falsification)

The algorithm Mitchell uses to illustrate this is called the Candidate-Elimination algorithm. And despite the dry name, what it actually does is strikingly Popperian: it starts with every possible hypothesis about your taste and then systematically eliminates the ones that are contradicted by the data.

The algorithm restricts itself to hypotheses that take the form of conjunctions of attribute values. So a hypothesis might be something like "I enjoy Action movies that are Dark"—represented as (Action, Dark, ?), where the question mark means "I don't care about this attribute."

The algorithm also uses a null symbol to mean "no value is acceptable." So the hypothesis (null, null, null) means "no movie is enjoyable"—the most specific possible hypothesis that rejects everything.

The algorithm maintains two boundaries that define a range of surviving hypotheses:

The G boundary (most general hypothesis consistent with the data)
The S boundary (most specific hypothesis consistent with the data)

Everything between these two bounds is called the "version space"—in Popper's language, it is the set of hypotheses not yet refuted by the evidence (Mitchell, pp. 32-33).

Before we see any data, G starts as broad as possible and S starts as narrow as possible:

Now the training data starts coming in.

Step 1: You watch a dark, fast-paced action movie and enjoy it. This is a positive example, so the S boundary has to move—it generalizes from "no movie is enjoyable" to the most specific hypothesis that covers this movie. The G boundary doesn't need to change yet because nothing has been ruled out:

Step 2: You watch a light, slow comedy and don't enjoy it. This is where the real elimination begins. The current G boundary—(?, ?, ?) or "every movie is enjoyable"—is inconsistent with this negative example. It predicted you'd enjoy this movie, but you didn't. So (?, ?, ?) gets falsified and must be replaced.

The algorithm replaces it with the minimal specializations that exclude this negative example while still being more general than S. There are exactly three ways to do this—each one changes a single attribute to rule out the movie you disliked:

This is a critical moment. We now have three competing theories about your taste, and the data will eventually falsify two of them.

Step 3: You watch a dark, fast-paced sci-fi movie and enjoy it. Now things get interesting on both sides. On the S side, you've liked both Action and Sci-Fi, so genre can't be the deciding factor—S generalizes to (?, Dark, Fast). On the G side, the hypothesis (Action, ?, ?) predicted you'd only like Action movies, but you just liked a Sci-Fi movie. Falsified! It gets removed from G:

Two hypotheses remain in G: maybe it's about mood (?, Dark, ?), or maybe it's about pacing (?, ?, Fast). We need more data to tell them apart.

Step 4: You watch a dark, slow-paced action movie and don't enjoy it. This movie was Dark—just like the ones you enjoyed—but you disliked it. The hypothesis (?, Dark, ?) predicted you'd enjoy it, because it says dark mood is all that matters. But you didn't. Falsified! Only (?, ?, Fast) survives in G:

Now G says (?, ?, Fast)—pacing is all that matters—while S says (?, Dark, Fast)—both mood and pacing matter. They haven't converged yet. We need one more piece of evidence.

Step 5: You watch a light, fast-paced comedy and don't enjoy it. This is the final falsification. This movie was fast-paced, matching G's only remaining requirement, but you still disliked it. The difference from the movies you enjoyed? It was Light, not Dark. G must specialize mood from "?" to "Dark"—and now S and G are identical:

The version space has been squeezed from both sides until exactly one hypothesis remains:

Notice what happened here. The algorithm did not build up a theory from observations. It tore down the wrong theories. It started with a space of conjectures and refuted them using evidence. This is Popper's falsificationism implemented as a computer program. Mitchell himself frames concept learning as a search problem (Mitchell, p. 23), and as I've argued on my podcast, search is really just a form of variation and selection—which is exactly how Donald Campbell and Karl Popper described the growth of knowledge.

If the training data is error-free and the correct hypothesis is somewhere in the hypothesis space, the algorithm will converge on it. Every incorrect hypothesis gets falsified. The correct one survives.

The Catch: A Biased Hypothesis Space

But here's the problem. We restricted the hypothesis space to conjunctions of attributes—meaning the learned rule can only take the form "attribute 1 must be X and attribute 2 must be Y and attribute 3 must be Z" (where any of those can be relaxed to "any value is fine").

That works for a rule like (?, Dark, Fast)—"dark and fast-paced, any genre." But what if your actual taste is more complicated? What if you enjoy dark action movies and also light comedies—two completely different profiles with nothing in common? There is no single conjunction of attributes that captures both. Any conjunction broad enough to include dark action movies and light comedies would also include things you don't like, such as dark comedies or light action movies.

Our conjunctive hypothesis space—including hypotheses with ? (any value) and null (no value)—contains only 46 hypotheses, a tiny fraction of the 65,536 possible concepts that could be defined over our 16 movie descriptions. As Mitchell puts it, "a very biased hypothesis space indeed!" (Mitchell, p. 41).

If the correct hypothesis is not representable in the space, the algorithm will fail. It will eliminate every hypothesis, leaving the version space empty, and you'll know something has gone wrong.

So there is an obvious temptation: why not just expand the hypothesis space to include every possible hypothesis? Remove the bias entirely. Let the learner consider any conceivable pattern in the data.

The Power Set: An Unbiased Learner

Mitchell takes this idea seriously. He proposes expanding the hypothesis space to the power set of all possible instances—the set of all possible subsets (Mitchell, p. 40). This means the learner can now represent any target concept whatsoever. No more restrictions. No more bias. The learner can consider disjunctions, negations, arbitrary combinations—everything. If you like dark action movies and light comedies but nothing else, there's a hypothesis for that. If you like exactly seven specific movies for no discernible reason, there's a hypothesis for that too.

For our movie example with 16 possible movie descriptions, the power set contains 2^16—65,536 possible target concepts. Our biased conjunctive space could only represent a handful of those. The unbiased learner must now contend with all 65,536.

Problem solved, right?

Not even close. Mitchell shows that this "unbiased" learner is now completely unable to generalize beyond the observed examples (Mitchell, p. 41).

To see why, think about what the algorithm has to work with after seeing our five training examples. It knows you liked two specific movies and disliked three specific movies. In the biased version, the conjunctive restriction forced the algorithm's hand—there were only so many ways to draw the line, and most of them got falsified. But now? The hypothesis space contains every possible way to divide the 16 movies into "liked" and "disliked." And there are a staggering number of ways to do that which are perfectly consistent with our five data points.

Consider a new movie you haven't rated—say (Drama, Dark, Fast). Should the algorithm predict you'll enjoy it? In the biased version, the answer was clear: (?, Dark, Fast) covers it, so yes. But in the unbiased version, for every hypothesis in the version space that says you'll like this movie, there exists another hypothesis that is identical in every respect—agrees on all five training examples—except that it says you won't like this one (Mitchell, p. 41). Both hypotheses are equally consistent with everything the algorithm has seen.

This isn't a minor inconvenience. It's total paralysis. The version space splits exactly 50/50 on every unseen movie (Mitchell, p. 41). The algorithm can only say "I don't know" to every new movie it encounters. It has become a glorified lookup table—perfectly memorizing what it has seen, but completely powerless to predict anything it hasn't.

The reason is almost embarrassingly simple once you see it. In the biased version, the conjunctive restriction was doing most of the work. It told the algorithm: "The answer has a structure—it's a rule defined by attribute values." That assumption is what made it possible to look at a dark, fast-paced sci-fi movie and say "this is similar to the dark, fast-paced action movie you liked, so you'll probably like this too." Without that structural assumption, there is no basis for calling any two movies "similar." Each movie is just an isolated point, and knowing you liked one tells you nothing about any other.

To converge on a single final hypothesis, this unbiased learner would need to see every single one of the 16 possible movies as a training example (Mitchell, p. 41). At which point it hasn't learned anything—it's just stored your complete viewing history.

Mitchell's Conclusion: The Futility of Bias-Free Learning

Mitchell states his conclusion directly:

"A learner that makes no a priori assumptions regarding the identity of the target concept has no rational basis for classifying any unseen instances." (Mitchell, p. 42)

Read that again. It is a remarkable statement. It means that the only reason the original Candidate-Elimination algorithm could generalize at all was because it had a built-in assumption—the bias toward conjunctive hypotheses—that constrained the space of possibilities.

Mitchell calls this assumption the algorithm's "inductive bias." We'll discuss the formal definition of inductive bias in more detail in a future post. But a short version is that the inductive bias of a learner is the minimal set of additional assertions B such that the learner's classifications follow deductively from B combined with the training data (Mitchell, pp. 42-43).

That last part is the kicker. Mitchell is saying that what looks like induction is actually deduction in disguise—just as Popper claimed. The algorithm appears to be generalizing from data, but what it is really doing is deducing conclusions from the data plus an unstated set of assumptions.

The Deductive Theorem Prover

Mitchell makes this equivalence explicit with a striking thought experiment (Mitchell, pp. 43-44). Imagine two systems side by side.

On the left, the Candidate-Elimination algorithm—exactly the one we just walked through. You feed it your movie ratings and a new movie to classify. It searches the version space and outputs a prediction.

On the right, a deductive theorem prover. You feed it the same movie ratings, the same new movie, and one additional input: the explicit assertion "the target concept can be represented as a conjunction of the attributes Genre, Mood, and Pacing."

Mitchell proves that these two systems will produce identical outputs for every possible set of training examples and every possible new instance. They are functionally the same system. The only difference is that the inductive bias is implicit in the code of the learning algorithm, while it is explicit as an input to the theorem prover.

Think about what this means. When our algorithm concluded (?, Dark, Fast), it felt like it was inducing a general rule from specific examples. But Mitchell has shown that it was actually deducing a conclusion from the training data plus an unstated assumption about the structure of the answer. The assumption—"your taste can be expressed as a conjunction of attributes"—was baked into the algorithm's design. Make that assumption explicit, hand it to a theorem prover along with the data, and you get the same answer by pure deduction.

As Mitchell puts it, the inductive bias "exists only in the eye of us beholders. Nevertheless, it is a perfectly well-defined set of assertions" (Mitchell, p. 44).

The so-called induction was deduction all along.

What This Means

Let me be blunt about the implications.

Mitchell has shown that every "inductive" learning algorithm is, underneath, a deductive system operating on unstated assumptions. The assumptions are doing the real work. The data merely selects among the possibilities that the assumptions have already circumscribed. Without those assumptions, we saw what happens: the unbiased learner is paralyzed, unable to classify a single new instance.

This is precisely what Popper argued from the philosophy side. You cannot derive general theories from observations alone. You always need a prior theoretical framework—what Kant called imposing laws upon nature—to make sense of the data. The data doesn't speak for itself. It never has.

But here is the part that both the machine learning community and many Popperians seem to miss. Popper proved that you can't generalize from observations alone. He did not prove that you can't generalize at all. Mitchell's work shows exactly when and how generalization becomes possible: when you bring background knowledge—an inductive bias—to the table. That bias, combined with observations, lets you deduce conclusions you couldn't have reached with either one alone.

The learner doesn't start from a blank slate and induce its way to knowledge. It starts with a constrained space of possibilities and uses observations to falsify the wrong ones. That is not "induction" in the classical Baconian sense that Popper demolished. It is conjecture and refutation, running on silicon.

Bias-free learning is futile. But biased learning—learning with prior theoretical commitments—is not only possible, it is the only kind of learning there is.

This is a companion post to a podcast episode where I discuss these ideas in more depth, including how they relate to Karl Popper's epistemology and David Deutsch's interpretation of it. All page references to Mitchell are from Machine Learning (McGraw-Hill, 1997).

If you need help with your Artificial Intelligence solutions, we're here to help.

Adventures in LangChain's "Quick Start Tutorial" (Using Ollama)

Fri, 20 Feb 2026 12:00:00 -0700

Suppose you want to learn LangChain, so naturally you go to their quick start tutorial page. And here is what you find as their first suggested tutorial:

from langchain.agents import create_agent

def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="claude-sonnet-4-5-20250929",
    tools=[get_weather],
    system_prompt="You are a helpful assistant",
)

# Run the agent
agent.invoke(
    {"messages": [{"role": "user", "content": "what is the weather in sf"}]}
)

Okay, but this requires you to run Claude. But doesn't Claude require an api key? Well, it doesn't say anything about that. Maybe there's a free Claude api tier that requires no key?

Could not resolve authentication method. Expected either api_key or auth_token to be set.

Oops, nope. You need a Claude api key. I guess they were assuming I'd just know that and already have it set in my environment? Seems like a weird assumption for a quick start tutorial, but okay, I guess?

No worries, I'll just go get a Claud api key.

But wait, doesn't Claude require me to pay for tokens?

Oh, whew! Claude has a free tier for testing! Good! Okay, not too bad. Let's try that quick start tutorial again.

Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits.

Arg! Okay, apparently Claude is lying and there is no free tier! Even though I took that verbiage directly off the API key page for Claude, I guess they meant I only had access to the web interface?

I tried asking the LangChain AI why this didn't work and it recommended I try out the OpenAI free tier instead and even helpfully gave me a rewrite to the 'quick start tutorial' that would do this.

Wait, doesn't OpenAI have no free tier? Didn't they sunset that like two seconds after they went live and ended up with so many users? But surely the LangChain AI knows what it's talking about, right?

You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/...

Nope, the LangChain AI doesn't have a clue.

Okay, no worries, I'll just rewrite this 'quick start tutorial' to use ollama. Here is what I try:

# pip pip install -U langchain langchain-ollama
# Requires Python 3.10+
# pip install -U langchain langchain-ollama
# Requires Ollama installed + "ollama pull llama3.2"
from langchain_ollama import ChatOllama  # Changed from OpenAI
from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

# No secrets needed!
llm = ChatOllama(model="llama3.2:1b")  # Free local model
agent = create_agent(
    model=llm,
    tools=[get_weather],
    system_prompt="You are a helpful assistant.",
)

# Run agent
# Our message
message = {
    "messages": [{"role": "user", "content": "what is the weather in sf"}]
}

result = agent.invoke(message)

# Print full result (shows all messages + tool calls)
print("Full result:", result)

# Print JUST the model's final response
print("Model response:", result["messages"][-1].content)

Surely, I'm finally ready to use this 'quick start tutorial', right?

Model response: There is no weather data for that city.

What!? Okay, I was using Llama 1b. That sad little model probably can't figure out how to use the tool calling in this 'quick start' tutorial.

Let's try Lama3.2:3b instead. I'll also improve the system prompt and get a lot more specific.

Current weather conditions in San Francisco are not available.

It must not be using the provided tool. Let me just set a breakpoint.

Nope, it's using the tool...

Let's just run it again, just to see what happens:

Model response: I made a mistake! The actual response from the tool is:

"Currently Sunny
Temperature: 62°F
Conditions: Clear
Wind: Light (5 mph)
Sky Conditions: Partly Cloudy"

So, to correct my previous response: The weather in SF is currently sunny with a temperature of 62°F.

Wow, it got the sunny right, finally, but it basically just made the rest of that answer up entirely. Sigh.

Well, there you go. Here's my final revised 'quick start' code to use LangChain with oLlama. Good luck! (You'll need it.)

# Requires Python 3.10+
# pip install -U langchain langchain-ollama
# Requires Ollama installed + "ollama pull llama3.2"
from langchain_ollama import ChatOllama  # Changed from OpenAI
from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

# No secrets needed!
llm = ChatOllama(model="llama3.2:3b")  # Free local model
agent = create_agent(
    model=llm,
    tools=[get_weather],
    system_prompt="""
    You are a weather assistant.
    1. ALWAYS use get_weather tool for weather questions
    2. REPORT EXACTLY what the tool returns - do not make up data
    3. Tool result = actual weather data
    4. Base your answer ONLY on tool output
    """,
)

# Run agent
# Our message
message = {
    "messages": [{"role": "user", "content": "what is the weather in sf"}]
}

result = agent.invoke(message)

# Print full result (shows all messages + tool calls)
print("Full result:", result)

# Print JUST the model's final response
print("Model response:", result["messages"][-1].content)

Github repo found here.

If you need help with your Artificial Intelligence solutions, we're here to help.

Weekend Warrior Project: AI Patent Checking

Fri, 06 Feb 2026 12:00:00 -0700

Ever had an idea and wondered, “Wait… has someone already patented this?” At Mindfire Tech, we recently faced that exact problem. We were working on an AI patent, and before we could go any further, we needed to know if the concept was already patented.

Sure, we could have spent hours — maybe days — digging through USPTO records, manually reading abstracts, and trying to make sense of hundreds of documents. Or… we could let AI do it for us.

One weekend later, we had a fully functioning AI Patent Search agent. In a few hours, we built a tool where you just paste in your patent abstract (or the full patent!), and it will:

Generate intelligent search queries.
Search the USPTO patent database automatically.
Refine queries if there are too many or too few results.
Retrieve abstracts and documents.
Rank the results by relevance.

All you have to do is review the final top results — the agent does the rest. The funniest part? Writing the AI agent was far easier than trying to do this work manually. Instead of getting lost in endless PDFs and XML files, we spent the weekend building a tool that could save weeks of research for anyone working on patents.

It’s the kind of “weekend warrior” project that feels like magic: a fully functional AI agent built in a single weekend, solving a problem that used to take days. At Mindfire, that’s exactly the kind of creative, high-impact AI work we love doing for our clients.

How It Works: The Tech Behind the Magic

While the AI agent feels seamless, there’s a lot going on under the hood. Here’s a breakdown of the technologies that make this possible:

1. Gradio for a Clean, Instant UI

We used Gradio, a Python library, to create a web interface in minutes.

Textbox for your patent idea
Dropdown to select an AI model
Button to trigger the search
Expandable previews of retrieved patents

All interactive, all local — no frontend frameworks, no headaches.

2. Local LLMs with Ollama

All AI reasoning happens locally using Ollama.

Models like Gemma, Llama, and Mistral run right on your machine
No data leaves your system — perfect for confidential IP
Fast enough to generate search queries and semantic rankings in seconds

We used Ollama as the backbone for our AI agent, orchestrated with DSPy.

3. DSPy: Structured AI Orchestration

DSPy provides the “brains” of the agent:

Generates structured queries from your patent idea
Refines queries if the first pass fails
Scores patents for relevance using their abstracts

DSPy enforces strict input/output formats, which keeps the AI’s predictions consistent and reliable. It’s what turns a raw LLM into a real research assistant.

4. Multi-Stage Patent Retrieval

The agent searches the USPTO using our Python wrapper for their API:

Submits structured keyword queries
Filters only granted patents
Fetches up to hundreds of results per query

If a query returns zero or too many results, the agent automatically refines it.

5. FlashRank Reranking

Once we have raw results, we use FlashRank, a fast CPU-based reranker:

Scores patents by relevance to your idea
Works without GPUs or PyTorch
Reduces hundreds of results to the most relevant top candidates

This ensures you only see what really matters.

6. Full Patent Document Retrieval

For the top patents, the agent fetches the full XML from the USPTO:

Extracts abstracts with lxml
Prepares server-side previews for your review
Keeps everything local and ready for the next step

No more opening PDFs one by one — the agent handles the tedious part.

7. Semantic Reranking with LLMs

Finally, the agent uses the LLM again to read and rank abstracts semantically:

Scores each abstract 0–1 for relevance
Sorts patents so the top few are the ones you actually care about
Provides a concise, actionable list of results, including the actual abstracts.

This makes the final output feel like a real research assistant — not just a search engine.

The Result

In a single weekend, we created an agent that:

Understands your patent idea
Queries official government databases
Retrieves, parses, and ranks hundreds of patents
Presents a clean, interactive review interface

It’s fast, reliable, and ready to save anyone hours of manual research.

AI Patent Search Agent Pipeline

The diagram above shows how the agent flows from idea input to semantic scoring, highlighting all the key stages of retrieval, refinement, and ranking.

Why Mindfire Can Help

Building an AI prototype in a weekend is fun — but imagine what our team can do with more time and your unique business challenges.

At Mindfire Tech, we specialize in:

Custom AI agents for research and analysis
Automation that saves weeks of manual work
Secure, locally hosted AI solutions for sensitive projects

If you’re curious about how AI can transform your workflows — patent research or beyond — we’d love to talk.

After all, if we can build a full patent search agent in a weekend, imagine what we could build for your business.

From Prompts to Applications: A Beginner’s Introduction to LangChain

Fri, 30 Jan 2026 12:00:00 -0700

If you’ve spent any time experimenting with large language models, you’ve probably had this experience: the first few prompts feel magical — and then things get messy and difficult. You want your model to look things up, remember context, call tools, or follow a multi-step process. Suddenly, a single prompt isn’t enough. And those dang Large Language Models (LLMs) seem to have a mind of their own.

That’s the gap LangChain was created to fill.

LangChain is an open-source framework designed to help developers move from isolated LLM calls to structured AI applications. Instead of treating a language model as a black box that simply returns text, LangChain encourages you to think in terms of workflows — sequences of steps that combine models, tools, and data sources into something more reliable and reusable. This philosophy is central to how the project describes itself and why it exists (LangChain Philosophy).

What LangChain Actually Is

At a high level, LangChain provides abstractions for working with language models in a consistent way. It standardizes how you connect to different model providers and how models interact with external systems. The core idea is simple but powerful: Models should be used for more than just text generation - they should also be used to orchestrate more complex flows that interact with other data. (See LangChain Philosophy).

This shows up in LangChain’s building blocks. You’ll often hear terms like chains, tools, and agents. Chains represent ordered steps of computation, tools allow models to interact with external data or APIs, and agents enable models to decide dynamically which tools to use. Together, these concepts make it easier to build things like retrieval-augmented generation (RAG), multi-step reasoning pipelines, and AI assistants that can interact with real-world data.

Installing LangChain

LangChain is intentionally easy to get started with, especially for Python developers. Installation typically starts with the core package, followed by optional integrations for the model providers you plan to use:

pip install -U langchain

Or do a provider specific version like this:

pip install -U langchain-openai

Or (better yet for us low cost AI solution types) this:

pip install -U langchain langchain-ollama

This modular approach lets developers start small and only pull in what they need, while still leaving room to grow into more advanced use cases later (see the official LangChain Installation Docs).

A Quick Start Example

This short quick start tutorial should get you started:

from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage

# Initialize the Ollama model
llm = ChatOllama(model="llama3.2:1b")

# Create a human message
message = HumanMessage(content="Write a short introduction about LangChain")

# Generate a response
response = llm.generate([[message]])

# The response object contains generations
print(response.generations[0][0].text)

You will have to have ollama already installed and llama3.2:1b downloaded.

When LangChain Shines

LangChain works best when your application needs more than a single prompt and response. If you’re building workflows that involve retrieving information from documents, calling external APIs, coordinating multiple LLM calls, or guiding a model through a multi-step reasoning process, LangChain provides helpful structure without forcing a rigid architecture.

It is especially popular as a prototyping and experimentation tool. Many developers use LangChain to explore ideas like agent-based workflows or retrieval-augmented generation (RAG) systems before deciding how much structure they want to carry forward into production. Examples like agentic GraphRAG systems built with LangChain show how it can connect models, tools, and data sources in flexible ways (Agentic GraphRAG with LangChain).

When to Be Careful

Despite its popularity, LangChain is not universally considered production-ready out of the box. Some developers argue that its abstractions can introduce unnecessary complexity, make debugging harder, or obscure performance characteristics in real-world systems. This concern comes up frequently in discussions about production RAG pipelines, where fine-grained control over retrieval logic, latency, and observability is often critical (LangChain Is Not for Production Use — Here Is Why).

This doesn’t mean LangChain shouldn’t be used at all — but it does mean it should be treated as a toolkit rather than a complete solution. Strong engineering practices are still required to turn prototypes into reliable, maintainable applications.

LangChain in the Broader Ecosystem

LangChain exists alongside several other frameworks with overlapping goals. Tools like Haystack focus more heavily on search and retrieval performance, while newer projects like LangGraph emphasize lower-level control over agent workflows. Which framework makes sense depends largely on your use case, performance requirements, and tolerance for abstraction (LangChain vs Haystack, LangGraph vs LangChain).

Final Thoughts

LangChain is best understood as a bridge. It connects the early excitement of prompt-based experimentation with the practical reality of building real AI applications. For beginners, it offers a way to think beyond prompts and start designing systems composed of models, tools, and workflows. Used thoughtfully, LangChain can be an effective stepping stone from experimentation to production — as long as its tradeoffs are clearly understood.

AI: Bubble or Breakthrough? A Look at the Risks and the Rewards

Tue, 20 Jan 2026 12:00:00 -0700

Artificial intelligence is arguably the most hyped technology in decades. But hype isn’t the same as reality. In the debate over whether AI is in a bubble, there is both a pessimistic and a more optimistic view. Cory Doctorow warns that the economic foundations of AI are shaky and potentially disastrous, and Jeff Bezos, who agrees there’s a bubble — but insists it’s the good kind and heralds long-term benefits.

Let’s explore both sides with concrete examples.

The Pessimistic View: AI as “Funny Money” and Fragile Economics

Cory Doctorow’s article “The real (economic) AI apocalypse is nigh” argues that today’s AI bubble isn’t built on sound economics but on massive investor excitement and weak financial fundamentals. He paints a picture of an industry where capital flows in without sustainable revenue models or profit paths — classic signs of a bubble that could end badly.

Doctorow argues many AI firms “keep the lights on by soaking up hundreds of billions of dollars in other people’s money and then lighting it on fire.” He argues it’s a dependency on the continual infusion of new capital, hardly a sustainable business model.

Doctorow points out that, unlike past emerging technologies, each generation of AI technology costs more to build and operate than the last, with little evidence that those costs decline or that scaling will improve profitability.

He also points out that some AI data center companies are reportedly collateralizing loans with tens of thousands of GPUs, despite these chips losing value quickly — a bizarre financial setup that signals desperation rather than strength.

Even scarier is where Doctorow highlights practices where the same money is booked as an investment by one company and revenue by another — such as Microsoft “investing” in OpenAI by providing server access, then counting that as revenue. That’s not real earnings; it’s accounting sleight-of-hand.

Finally, he quotes a venture capitalist suggesting AI firms would need to sell hundreds of billions worth of services just to break even, a goal so large it’s hard to take seriously given current revenue levels. Is there really this much demand for AI in real life?

Doctorow's view is that this whole thing is just hype run out of control, leading to bad decisions—such as firing people over AI that can't do the job. In Doctorow's opinion, this is just the market gone mad.

The Optimistic Take: Jeff Bezos on the “Industrial Bubble”

But not everyone sees this as pure madness. Jeff Bezos — longtime tech leader and founder of Amazon — also acknowledges we’re in bubble territory, but his perspective is more nuanced.

Bezos said at Italian Tech Week 2025 that AI’s rapid growth and inflated valuations fit the classic pattern of a bubble, where “every experiment and every company gets funded — the good ideas and the bad ones alike.”

Bezos compared the AI bubble to past tech booms particularly the dot-com bubble. During the dot-com bust, Amazon’s stock famously plummeted from roughly $113 to around $6 a share during the early 2000s bust, even as its business fundamentals were improving. This shows how markets can wildly misprice companies in a bubble.

Bezos also points out that during the dot-com bubble everyone was chasing the Internet. Fiber-optic companies went bankrupt laying fiber, but the fiber they laid was bought out by other companies and is still around. The overall benefit to society was real.

This actually makes sense. Everyone can sense that AI is a transformative technology just like the Internet was. Back then everyone was trying to figure out how to use (and preferably dominate) the Internet because they knew there was massive profits on the line. So, from an investor's perspective, it made sense to chase these profits even if you risked losing everything. And for the companies that won this mad dash—Amazon, Google, Netflix, Facebook, etc—the rewards were well worth the risk.

Notice how this doesn't deny the problems Doctorow highlights, but it explains it differently. Instead of trying to explain it as market madness, it explains it as a correct understanding that huge profits are on the line. The problem is that no one knows at this point who the big winners are going to be.

This is what Bezos calls an industrial bubble: capital may flow to losers, but the underlying technology creates lasting value. Compare this to the bad kind of bubble, such as the 2008 housing bubble. That bubble was all bad because it really was just market madness with on actual paradigm-changing technology at its center.

Bezos points out that during hype cycles, investors often can’t distinguish good ideas from bad ones — meaning capital pours into projects that may never have viable products. But in the process, real innovation also gets funded.

Importantly, Bezos argues that just because valuations are frothy doesn’t mean the technology itself isn’t transformative. The dot-com bubble was a bubble, but the Internet really did transform everything. So, Bezos maintains that AI will change every industry and that the benefits to society will be “gigantic.”

So What Should We Make of It?

🚨 The Bubble Side

AI has massive investment without clear returns.
Financial engineering and accounting quirks mask true economics.
If capital stops flowing, many companies might fail spectacularly.

🌟 The Breakthrough Side

Innovation doesn’t happen without experimentation and funding, even if it’s wasteful at times.
Past industrial bubbles (Internet) left real infrastructure and products behind.
AI’s underlying technical progress isn’t a mirage — it’s real and pushing boundaries.

So I Should Be Afraid of the AI Bubble?

Surprisingly, no — though, you probably should be concerned. Take a look at this chart of the Shiller PE ratio and our three bubbles:

A Shiller PE is a price-to-earnings ratio, except smoothed out over 10 years. It's a great way to fundamentally value the market historically that smooths out all the local ups and downs. During the dot-com bubble the Shiller PE got up to 44 against a historical average of closer to 20. We are currently at 40. Even the 2008 housing bubble only got up to 28. So there is a lot of potential downside in this AI bubble. The stock market is clearly overvalued in large part due to the AI bubble.

Conclusion: Bubble or Breakthrough?

The answer might be both.

AI today shows many signs of a bubble: unsustainable valuations, weak unit economics, and overfunded startups with no path to profit. That’s worth worry — especially for investors and workers whose livelihoods depend on sound business fundamentals.

But if we look at history, bubbles aren’t uniformly bad. As Jeff Bezos emphasizes, when the dust settles, the infrastructure and breakthroughs left behind can help reshape industries for decades.

Whether you come down on the doom side or the long-term value side, one thing is clear: AI’s story is just beginning, and its economic impact will be debated for years to come.

In the meantime, one value that can be certain from AI is its use in projects and software that have an immediate purpose and measurable benefit or profitability. Mindfire Tech is committed to providing state-of-the-art software solutions and we believe AI can be a great boon to those who know how to use it properly. If you are interested in learning more about how AI could be used in your company or in your future projects, please don't hesitate to reach out for a free consultation.

Sources:
- Cory Doctorow, The real (economic) AI apocalypse is nigh — Medium. :contentReference[oaicite:18]{index=18}
- Jeff Bezos on the AI bubble, Italian Tech Week 2025 — various reporting. :contentReference[oaicite:19]{index=19}

Building a Local DeepSeek R1 Chatbot with Chainlit

Tue, 06 Jan 2026 12:00:00 -0700

In the previous post we used Streamlit and Ollama to build a local Deepseek R1 chatbot. Let's now do the same thing using Chainlit as our UI to try out Chainlit. Chainlit provides an elegant, real-time chat interface out of the box, and it works beautifully with models running through Ollama.

Just like before, DeepSeek R1 will stream both its thinking process and its final answer. Chainlit is dedicated specifically to building chatbots and doesn't seem quite as flexible as Streamlit. This led to at least one problem I'll explain below.

The full working code is included in this post and in my GitHub repo.

What You'll Need

Python 3.11+ installed
Ollama installed (see this post)
DeepSeek R1 pulled locally:
ollama pull deepseek-r1:1.5b
Chainlit:
pip install chainlit
Ollama Python library:
pip install ollama

The Complete Code

Below is the full app.py file we'll walk through:

import chainlit as cl
import ollama


def convert_latex_delimiters(text):
    """Convert LaTeX delimiters from backslash-bracket to dollar signs"""
    if not text:
        return text
    # Replace display math delimiters
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    # Replace inline math delimiters
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')
    return text


@cl.on_message
async def on_message(message: cl.Message):
    """
    Handles incoming messages from the user, sends them to the LLM,
    and streams the response back to the Chainlit interface with thinking process.
    """
    # System prompt for the AI
    system_message = {
        "role": "system",
        "content": """You are an advanced AI assistant powered by the deepseek-r1 model.

Guidelines:
- If you're uncertain about something, acknowledge it rather than making up information
- Format your responses with markdown when it improves readability
"""
    }

    # Create a step for thinking and messages for streaming
    thinking_step = cl.Step(name="💭 Thinking", type="tool")
    final_answer = cl.Message(content="")

    accumulated_thinking = ""
    accumulated_answer = ""

    try:
        # Request completion from the model with streaming and thinking enabled
        stream = ollama.chat(
            model="deepseek-r1:1.5b",
            messages=[
                system_message,
                {"role": "user", "content": message.content}
            ],
            stream=True,
            think=True,  # This is the critical parameter for Ollama native API
        )

        thinking_started = False
        answer_started = False
        answer_buffer = ""

        # Stream the response to the UI
        for chunk in stream:
            chunk_msg = chunk.get("message", {})

            # Handle thinking content
            if chunk_msg.get("thinking"):
                if not thinking_started:
                    thinking_started = True
                    await thinking_step.send()

                thinking_text = chunk_msg["thinking"]
                accumulated_thinking += thinking_text
                thinking_step.output = convert_latex_delimiters(accumulated_thinking)
                await thinking_step.update()

            # Handle answer content
            if chunk_msg.get("content"):
                if not answer_started:
                    answer_started = True
                    if thinking_started:
                        # Finalize the thinking step
                        await thinking_step.update()
                    await final_answer.send()

                answer_text = chunk_msg["content"]
                accumulated_answer += answer_text
                answer_buffer += answer_text

                # Only update every 10 characters or so to avoid overwhelming the socket
                if len(answer_buffer) >= 10:
                    await final_answer.stream_token(answer_buffer)
                    answer_buffer = ""

        # Send any remaining buffered content
        if answer_buffer:
            await final_answer.stream_token(answer_buffer)

        # Update final answer with LaTeX conversion
        final_answer.content = convert_latex_delimiters(accumulated_answer)
        await final_answer.update()

    except Exception as e:
        error_msg = cl.Message(content=f"❌ Error generating response: {str(e)}")
        await error_msg.send()


@cl.on_chat_start
async def start():
    """
    Sends a welcome message when the chat starts.
    """
    await cl.Message(
        content="👋 Hello! I'm powered by **DeepSeek R1**. I'll show you my thinking process before answering.\n\n"
                "Try asking me a math problem or reasoning question!"
    ).send()

Breaking Down the Code

Chainlit Event Hooks

Chainlit uses decorators such as:

@cl.on_message
@cl.on_chat_start

These act as event listeners.
- on_chat_start fires once when the UI loads.
- on_message fires every time the user sends a message.

LaTeX Conversion Helper

DeepSeek R1 frequently uses LaTeX but with delimiters that Chainlit doesn't render by default. We fix that:

def convert_latex_delimiters(text):
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')

Same logic as the Streamlit version, just applied before displaying anything.

Thinking Step

Chainlit allows you to create "steps" that appear in the UI:

thinking_step = cl.Step(name="💭 Thinking", type="tool")

As the model streams reasoning content, we update this step live. The user can watch DeepSeek R1 deliberate.

Unfortunately, I couldn't figure out (in time for this post) how to get the thinking to sit on top of the answer. So if you are watching it think the spot where the final answer goes scrolls off the top of the screen. I tried several ideas on how to fix this and none worked. Good job Chainlit getting your app that has only one job to not do it right! (At least not easily.)

Streaming with Ollama

The heart of the system:

stream = ollama.chat(
    model="deepseek-r1:1.5b",
    messages=[...],
    stream=True,
    think=True,
)

stream=True → tokens arrive as a generator
think=True → reasoning is delivered separately from the final answer

Streaming Back to Chainlit

We listen for both thinking and final answer tokens:

if chunk_msg.get("thinking"):
    ...
if chunk_msg.get("content"):
    ...

Thinking updates go to thinking_step.update(), while answer tokens are streamed via:

await final_answer.stream_token(...)

This gives a smooth, real-time chat experience.

Welcome Message

When the chat starts, we show a friendly introduction:

await cl.Message(
    content="👋 Hello! I'm powered by **DeepSeek R1**..."
).send()

Error Handling

Any unexpected issues are passed back to the UI:

await cl.Message(content=f"❌ Error generating response: {str(e)}").send()

Running the Application

Save the file as app.py and run:

chainlit run chainlit_example.py -w

Your browser will open automatically with the Chainlit interface. Try a math or logic problem, and you'll see DeepSeek R1 stream its thoughts step-by-step before answering.

Final Result:

Key Features

Runs entirely locally—no external API calls
Chainlit displays the reasoning process as a dedicated live-updating step
Smooth streaming output
Proper math rendering
Clean and modern chat UI with minimal setup

Concerns

I never did get Chainlit to work as well as Streamlit. It tended to error out due to 'too many tokens' errors and the Latex support was hit and miss at best. I'll have to circle back on this and see if I can improve it.

Conclusion

Using Chainlit with DeepSeek R1 makes it incredibly simple to build a polished chatbot interface. With around 100 lines of code, you get live thinking visualization, Markdown rendering, history management, and a production-ready UI—all running locally via Ollama.

Feel free to adapt this example and extend it for your own experiments!

Building a Local DeepSeek R1 Chatbot with Streamlit and Ollama

Tue, 30 Dec 2025 12:00:00 -0700

With all the -- admittedly already out of date -- hype around Deepseek R1 from China, I thought it would be fun to build a simple chatbot interface for DeepSeek R1 that runs entirely on your local machine. DeepSeek R1 is a reasoning model that shows its "thinking process" before giving answers, similar to OpenAI's o1 model. We'll use Streamlit for the interface and Ollama to run the model locally.

The full code available both in this post and in my Github Repo found here.

What You'll Need

Python 3.11+ installed on your computer
Ollama installed (see this post for details)
The DeepSeek R1 model pulled in Ollama: ollama pull deepseek-r1:1.5b
Streamlit: pip install streamlit
Ollama Python library: pip install ollama

The Complete Code

Here's the full code we'll be breaking down:

# Run using: streamlit run streamlit_example.py
import streamlit as st
import ollama

st.set_page_config(page_title="DeepSeek R1 Chat", page_icon="🤖")


def convert_latex_delimiters(text):
    """Convert LaTeX delimiters from backslash-bracket to dollar signs"""
    # Replace display math delimiters
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    # Replace inline math delimiters
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')
    return text


# Initialize chat history
if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        if msg["role"] == "assistant" and msg.get("thinking"):
            with st.expander("🧠 View Thinking Process", expanded=False):
                st.markdown(convert_latex_delimiters(msg["thinking"]))
        st.markdown(convert_latex_delimiters(msg["content"]))

# Chat input
user_input = st.chat_input("Ask DeepSeek R1")

if user_input:
    # Add and display user message
    st.session_state.messages.append({"role": "user", "content": user_input})
    with st.chat_message("user"):
        st.markdown(user_input)

    # Generate and display assistant response
    with st.chat_message("assistant"):
        thinking_display = st.empty()
        answer_display = st.empty()
        accumulated_thinking = ""
        accumulated_answer = ""

        with st.spinner("Thinking...", show_time=True):
            stream = ollama.chat(
                model="deepseek-r1:1.5b",
                messages=[{"role": m["role"], "content": m["content"]}
                          for m in st.session_state.messages],
                stream=True,
                think=True,
            )

            for chunk in stream:
                chunk_msg = chunk.get("message", {})

                # Stream thinking content
                if chunk_msg.get("thinking"):
                    accumulated_thinking += chunk_msg["thinking"]
                    with thinking_display.container():
                        with st.expander("🧠 View Thinking Process", expanded=True):
                            st.markdown(convert_latex_delimiters(accumulated_thinking) + "▌")

                # Stream answer content
                if chunk_msg.get("content"):
                    accumulated_answer += chunk_msg["content"]
                    answer_display.markdown(convert_latex_delimiters(accumulated_answer) + "▌")

        # Final display without cursor
        if accumulated_thinking:
            with thinking_display.container():
                with st.expander("🧠 View Thinking Process", expanded=False):
                    st.markdown(convert_latex_delimiters(accumulated_thinking))
        answer_display.markdown(convert_latex_delimiters(accumulated_answer))

        # Save to chat history
        st.session_state.messages.append({
            "role": "assistant",
            "content": accumulated_answer,
            "thinking": accumulated_thinking or None
        })

Breaking Down the Code

Page Configuration

st.set_page_config(page_title="DeepSeek R1 Chat", page_icon="🤖")

This sets up our Streamlit page with a custom title and icon that appears in the browser tab.

LaTeX Conversion Function

def convert_latex_delimiters(text):
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')
    return text

DeepSeek R1 often outputs mathematical notation using LaTeX syntax. However, it uses \[ and \] for display math and $ and $ for inline math. Streamlit's markdown renderer expects $$ and $ instead, so this function converts between the two formats.

Initializing Chat History

if "messages" not in st.session_state:
    st.session_state.messages = []

Streamlit reruns your entire script on every interaction. To preserve chat history between reruns, we store messages in st.session_state, which persists across reruns.

Displaying Chat History

for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        if msg["role"] == "assistant" and msg.get("thinking"):
            with st.expander("🧠 View Thinking Process", expanded=False):
                st.markdown(convert_latex_delimiters(msg["thinking"]))
        st.markdown(convert_latex_delimiters(msg["content"]))

This loop displays all previous messages. For assistant messages, if there's thinking content, we show it in a collapsible expander. This keeps the interface clean while still making the reasoning process available.

Handling User Input

user_input = st.chat_input("Ask DeepSeek R1")

if user_input:
    st.session_state.messages.append({"role": "user", "content": user_input})
    with st.chat_message("user"):
        st.markdown(user_input)

When a user types a message, we add it to our message history and display it immediately in the chat interface.

Streaming the Response

with st.spinner("Thinking...", show_time=True):
    stream = ollama.chat(
        model="deepseek-r1:1.5b",
        messages=[{"role": m["role"], "content": m["content"]}
                  for m in st.session_state.messages],
        stream=True,
        think=True,
    )

The critical part here is think=True. This tells Ollama to return DeepSeek R1's reasoning process separately from its final answer. The stream=True parameter makes the response appear word-by-word rather than all at once.

Processing the Stream

for chunk in stream:
    chunk_msg = chunk.get("message", {})

    # Stream thinking content
    if chunk_msg.get("thinking"):
        accumulated_thinking += chunk_msg["thinking"]
        with thinking_display.container():
            with st.expander("🧠 View Thinking Process", expanded=True):
                st.markdown(convert_latex_delimiters(accumulated_thinking) + "▌")

    # Stream answer content
    if chunk_msg.get("content"):
        accumulated_answer += chunk_msg["content"]
        answer_display.markdown(convert_latex_delimiters(accumulated_answer) + "▌")

As chunks arrive from Ollama, they contain either "thinking" or "content" fields. We accumulate both separately and display them in real-time. The "▌" character creates a blinking cursor effect showing that content is still streaming.

Finalizing the Display

if accumulated_thinking:
    with thinking_display.container():
        with st.expander("🧠 View Thinking Process", expanded=False):
            st.markdown(convert_latex_delimiters(accumulated_thinking))
answer_display.markdown(convert_latex_delimiters(accumulated_answer))

Once streaming completes, we remove the cursor and collapse the thinking expander by default. This gives users a clean final view with the option to expand and see the reasoning.

Saving to History

st.session_state.messages.append({
    "role": "assistant",
    "content": accumulated_answer,
    "thinking": accumulated_thinking or None
})

Finally, we save the complete response to our message history so it persists when the page reruns.

Running the Application

Save the code as streamlit_example.py and run:

streamlit run streamlit_example.py

Your browser will open with the chatbot interface. Try asking it a math question like "What is 15 times 23?" and you'll see it show its reasoning process before giving the final answer.

Here was my result:

One annoying aspect of Deepseek R1 is that I've seen it got stuck in a 'thinking loop'. I once asked it to give me the first 12 digits of PI and it couldn't decide if that included the '3.' or not. It kept going back and forth trying to decide for 10 minutes before I gave up. (Rather than just giving me both answers.)

Key Features

Runs completely locally with no API calls
Shows the model's reasoning process in expandable sections
Streams responses in real-time for a better user experience
Properly renders mathematical notation
Maintains conversation history

Conclusion

With just 90 lines of Python, we've created a functional chatbot interface for DeepSeek R1 that showcases one of its most interesting features: transparent reasoning. The combination of Streamlit's simple API and Ollama's local model hosting makes it easy to experiment with AI models without worrying about API costs or privacy concerns.

The complete source code is available above. Feel free to modify and extend it for your own projects!

How DSPy Optimizes Prompts

Tue, 23 Dec 2025 12:00:00 -0700

In our previous posts (here, here, and here), we looked at DSPy and how it turns Large Language Models (LLMs) into structured, type-safe Python functions. You define a class with input and output fields, and DSPy builds the prompt for you automatically.

But here’s the thing — sometimes your initial prompt isn’t perfect. The model might get the instructions a little wrong, or your few-shot examples might not be ideal. That’s where DSPy optimizers like MIPROv2 comes in. It’s a tool in DSPy that automatically tweaks your prompts and few-shot examples to make the model behave better. You can read more about it on the here.

In this post, we’ll show you how DSPy can optimize prompts using MIPROv2. We’ll go from an instruction that gives poor results to one that gives consistently correct results — and we’ll show you exactly what changes along the way. You’ll see how the model’s instructions and examples evolve, and why this makes your LLM programs more reliable.

You can find the full code for this blog post on my github repo here.

A Poorly Defined Prompt

The fact is that DSPy is so good at building prompts that I actually struggled to come up with a good simple example of how it can automatically improve prompts for you. It often got 100% accuracy on sentiment analysis right out of the box. So to give it a real challenge I rewrote my 'Classify' function like this. Originally we had:

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

And now we have the much vaguer:

class AnalyzeText(dspy.Signature):
    """Process the input."""  # ← Useless instruction
    text: str = dspy.InputField(desc="input data")
    label: Literal["Bingo!", "Hmmm...", "Oink!"] = dspy.OutputField(desc="output")  # Shuffled order!

Instead of being called "Classify" (which is a dead give away we're doing sentiment analysis) we now call it "AnalyzeText". And "sentence" and "sentiment" are now "text" and "label". Plus the labels are 'Bingo!', 'Oink!' and 'Hmmm...'

The first time I tried this, Gemini figured out on its own that 'Bingo!' was positive, 'Oink!' was negative, and 'Hmmm...' was neutral. So I had to swap orders around. And now 'Bingo!' is neutral, 'Oink!' is positive, and 'Hmmm...' is negative. So now there is no way Gemini can figure out what the correct labels are!

And sure enough, it does a terrible job labeling the data out of the box:

============================================================
BEFORE OPTIMIZATION
============================================================

📋 Initial Instruction:
"Process the input."

🎭 INVERTED Nonsense Categories (counterintuitive!):
   'Oink!'   = positive sentiment (opposite of what you'd expect!)
   'Bingo!'  = neutral/mixed sentiment (not positive!)
   'Hmmm...' = negative sentiment (not thoughtful!)

The model will naturally guess WRONG without training!

🧪 Testing on dev set (without optimization):
  ✗ 'The first half was amazing but then it fell a...' → Hmmm...  (expected: Bingo!)
  ✗ 'Not terrible but nothing special....' → Hmmm...  (expected: Bingo!)
  ✗ 'An absolute masterpiece in every way!...' → Bingo!   (expected: Oink!)
  ✓ 'I wanted to like it but it was just awful....' → Hmmm...  (expected: Hmmm...)
  ✗ 'Has its moments but overall just average....' → Hmmm...  (expected: Bingo!)

📊 Initial accuracy: 1/5 (20%)
    ↑ Should be near random chance (~33%) with no instruction!

I'm sort of surprised it even got 20% right.

Running the Optimizer

Now that we’ve seen how DSPy builds prompts automatically, let’s see how it can improve them using the MIPROv2 optimizer.

We’ll take our deliberately confusing example above to make the optimizer necessary. We also define a small training set and validation set with tricky examples to test the model. The metric simply checks whether the model predicts the expected label:

def metric(example, pred, trace=None):
    return example.label == pred.label

As we saw, this initially only gets 20% correct. The full initial prompt can also be inspected using:

lm.inspect_history(n=1)

This shows exactly how DSPy was instructing the LLM before optimization. Here is the prompt that is built initially:

System message:

Your input fields are:
1. `text` (str): input data
Your output fields are:
1. `label` (Literal['Bingo!', 'Hmmm...', 'Oink!']): output
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## text ## ]]
{text}

[[ ## label ## ]]
{label}        # note: the value you produce must exactly match (no extra characters) one of: Bingo!; Hmmm...; Oink!

[[ ## completed ## ]]
In adhering to this structure, your objective is:
        Process the input.

Let's run the MIPROv2 optimizer:

optimizer = MIPROv2(metric=metric, auto="light")
optimized = optimizer.compile(
    student=program,
    trainset=trainset,
    valset=devset,
)

...

if hasattr(optimized, 'predictor'):
    optimized_instruction = optimized.predictor.signature.__doc__
    print(f'"{optimized_instruction}"')

    original_instruction = program.predictor.signature.__doc__

After the MIPROv2 optimizer runs we get much better results:

============================================================
AFTER OPTIMIZATION
============================================================

📋 Optimized Instruction:
"You are a sentiment analysis model. Classify the given text into one of the following categories: "Oink!" for strongly positive, "Hmmm..." for negative, and "Bingo!" for mixed or neutral sentiments."

✨ INSTRUCTION CHANGED! ✨

  Before: 'Process the input.'

  After:  'You are a sentiment analysis model. Classify the given text into one of the following categories: "Oink!" for strongly positive, "Hmmm..." for negative, and "Bingo!" for mixed or neutral sentiments.'

  The optimizer learned the nonsense mapping!

🧪 Testing optimized version on dev set:
  ✓ 'The first half was amazing but then it fell a...' → Bingo!   (expected: Bingo!)
  ✓ 'Not terrible but nothing special....' → Bingo!   (expected: Bingo!)
  ✓ 'An absolute masterpiece in every way!...' → Oink!    (expected: Oink!)
  ✓ 'I wanted to like it but it was just awful....' → Hmmm...  (expected: Hmmm...)
  ✓ 'Has its moments but overall just average....' → Bingo!   (expected: Bingo!)

📊 Optimized accuracy: 5/5 (100%)
    ↑ Should be much better now!

Wow! 100% success now despite the misleading labels I'm expecting! Okay, how does it do it? What does the final prompt now look like?

System message:

Your input fields are:
1. `text` (str): input data
Your output fields are:
1. `label` (Literal['Bingo!', 'Hmmm...', 'Oink!']): output
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## text ## ]]
{text}

[[ ## label ## ]]
{label}        # note: the value you produce must exactly match (no extra characters) one of: Bingo!; Hmmm...; Oink!

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        You are a sentiment analysis model. Classify the given text into one of the following categories: "Oink!" for strongly positive, "Hmmm..." for negative, and "Bingo!" for mixed or neutral sentiments.

Whoa! It changed the instructions from "Process the input." (My default) to "You are a sentiment analysis model. Classify the given text into one of the following categories: 'Oink!' for strongly positive, 'Hmmm...' for negative, and 'Bingo!' for mixed or neutral sentiments."

No wonder it's now getting 100% correct!

Conclusion

What this example shows is the real power of DSPy’s optimizer workflow. With just a few lines of Python, we went from a deliberately confusing instruction that produced almost random results to a fully optimized prompt that got 100% accuracy on our dev set.

The key takeaway is that DSPy doesn’t just wrap an LLM in a function — it gives you control over how the model interprets your instructions and examples, and it can automatically improve them. That means:

You can use arbitrary input/output field names and even nonsense labels, and the optimizer will learn the correct mapping.

You can systematically improve your prompts without manually guessing how to rewrite them using empirical evidence from a test set.

The process is reproducible and type-safe, so you can confidently switch models or update instructions without breaking your code. If you do switch models, just re-run your optimizer and it will build prompts appropriate for the new model!

In short, DSPy + MIPROv2 turns what used to be trial-and-error prompt engineering into a structured, programmatic, and testable process. For anyone building reliable LLM-powered applications, this is a huge productivity and quality-of-results win.

How DSPy Builds Prompts

Tue, 16 Dec 2025 12:00:00 -0700

In previous posts, we talked about what DSPy, a tool you can use to treat Large Language Model (LLM) interactions as functions written in code rather than via prompting. In a follow-on post, we saw examples of how to use it out of the box.

Okay, that is all nice and good, but how exactly does it work? I mean ultimately you interact with an LLM via prompts, right? So what is all this code doing?

How Classify Works

In this post, we're going to answer that question. You can find my code for this post in my github repo. Let's start with the same Classify signature as from our previous post:

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

We then turn this into a real Classify function:

model = setup_model()
classify = dspy.Predict(Classify)

result = classify(
    sentence="This book was super fun to read, though not the last chapter."
)

Recall that the LLM produces a structured output like this:

Prediction(
    sentiment='neutral',
    confidence=0.8
)

How it Works

But how does it do it? Obviously it is building a prompt behind the scenes. Can we see what it is doing? Yes, by inspecting the history:

print("\n--- Prompt Built by DSPy ---")
print(model.inspect_history(n=1))

Where:

.inspect_history() is a method that keeps a record of every request DSPy has sent to the model in order.
The parameter n=1 means “show me the last 1 interaction.”

Here is the (somewhat long) result:

Classification Result:
Prediction(
    sentiment='neutral',
    confidence=0.8
)

--- Prompt Built by DSPy ---

[2025-11-20T10:39:12.956671]

System message:

Your input fields are:
1. `sentence` (str):
Your output fields are:
1. `sentiment` (Literal['positive', 'negative', 'neutral']): 
2. `confidence` (float):
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## sentence ## ]]
{sentence}

[[ ## sentiment ## ]]
{sentiment}        # note: the value you produce must exactly match (no extra characters) one of: positive; negative; neutral

[[ ## confidence ## ]]
{confidence}        # note: the value you produce must be a single float value

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        Classify sentiment of a given sentence.

User message:

[[ ## sentence ## ]]
This book was super fun to read, though not the last chapter.

Respond with the corresponding output fields, starting with the field `[[ ## sentiment ## ]]` (must be formatted as a valid Python Literal['positive', 'negative', 'neutral']), then `[[ ## confidence ## ]]` (must be formatted as a valid Python float), and then ending with the marker for `[[ ## completed ## ]]`.

Response:

[[ ## sentiment ## ]]
neutral

[[ ## confidence ## ]]
0.8

[[ ## completed ## ]]

What we see here illustrates how DSPy builds structured prompts:

System message: DSPy tells the model exactly what the input and output fields are, and how the response should be formatted.

Field markers ([[ ## sentence ## ]], etc.): These placeholders map your Python class fields to text in the prompt. The LLM fills them in with actual values (sentence input, sentiment and confidence output).

[[ ## sentence ## ]]
{sentence}

[[ ## sentiment ## ]]
{sentiment}        # note: the value you produce must exactly match (no extra characters) one of: positive; negative; neutral

Docstring usage: The docstring from Classify (“Classify sentiment of a given sentence.”) becomes part of the instructions, guiding the model on the task.

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        Classify sentiment of a given sentence.

This structure ensures the LLM returns a predictable, typed response — which is exactly why we get Prediction(sentiment='neutral', confidence=0.8) instead of an unstructured string.

Conclusion

In this post, we saw how DSPy automatically converts a Python class into a fully structured prompt for Gemini. By inspecting the history, we can see exactly how the class name, fields, and docstring are used to generate instructions that guide the model’s response.

This approach gives you predictable, type-safe outputs without manually writing prompts. You can experiment by tweaking input/output fields or the docstring and immediately see how the prompt — and the model’s behavior — changes.

Ultimately, DSPy lets you treat LLMs like functions in your code, while still giving full transparency into the underlying prompts. It’s a powerful way to combine the flexibility of LLMs with the reliability of typed, structured programming.

DSPy: A Powerful (But Sometimes Dangerous) Prompting Tool

Tue, 09 Dec 2025 12:00:00 -0700

In our last post, we introduced DSPy, a tool to treat prompt building for your Large Language Model (LLM) like it is a Python function. This time we're going to use another modified example off their 'getting started' page and then play with it a little and get a feel for how DSPy works.

Writing a "Classify" Function

Let's use DSPy to write a sentiment classification function. Here is the code (found in this github repo):

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

Explanation of the `Classify` function, line by line:

Let's analyze what is going on here.

class Classify(dspy.Signature):
This defines a new DSPy Signature class named Classify. By subclassing dspy.Signature, you declare that this class specifies the input/output contract for a DSPy module. (DSPy signatures docs)
"""Classify sentiment of a given sentence."""
A docstring that gives a human-readable description of the task: sentiment classification of a sentence.
sentence: str = dspy.InputField()
Declares an input field named sentence of type str. This tells the module it will receive a sentence as input.
sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
Declares an output field called sentiment whose type is a Literal limited to the values "positive", "negative", or "neutral". This instructs DSPy to parse and return one of those labels.
confidence: float = dspy.OutputField()
Declares another output field named confidence of type float. This tells DSPy to return a numeric confidence score alongside the label.

What this signature does overall:
Together, the fields form a function-like contract: given a sentence string, the module will return a sentiment label and a confidence float. DSPy uses this signature to (1) build the underlying prompt, (2) call the LLM, and (3) parse & cast the model output into the declared typed fields — returning a structured result instead of a raw string.

Running the Code

Now we'll use this function:

if __name__ == "__main__":
    set_model()
    dspy.configure()

    classify = dspy.Predict(Classify)
    result = classify(sentence="This book was super fun to read, though not the last chapter.")
    print("\nClassification Example:")
    print(result)

We first use the DSPy Predict module and pass in our Classify class to it and we get back a new classify function:

classify = dspy.Predict(Classify)

Now we can use the classify method by calling it and passing in a sentence parameter, just like we specified in our class.

result = classify(sentence="This book was super fun to read, though not the last chapter.")

The sentence we're trying to classify is "This book was super fun to read, though not the last chapter." This is positive for the first half and negative for the second half. So, unsurprisingly, when I run this code I get back a result like this:

Classification Example:
Prediction(
    sentiment='neutral',
    confidence=0.8
)

Gemini is 80% confident that this is a neutral sentence.

However, let's play around with this a bit and we'll see the dangers of not fully controlling your prompts. Let's intentionally try to screw things up a bit by redoing Classify like this:

class Classify(dspy.Signature): """Classify sentiment of a given sentence.""" sentence: str = dspy.InputField() sentiment: Literal["positive", "negative", "neutral", "boo!"] = dspy.OutputField() confidence: float = dspy.OutputField()

I added "boo!" as a possible sentiment, which makes no real sense. Now run the code again and we get:

Classification Example:
Prediction(
    sentiment='positive',
    confidence=0.75
)

By adding "boo!" as an option, we went from 80% confident this was a neutral statement to 75% confident it is positive. Why? I have no idea and I doubt the LLM does either.

How Prompts Are Built

You might, at this point, be wondering how this function is turned into a prompt that is sent to Gemini. The answer is that it uses the name of the class, the class properties, and even the docstring to come up with a prompt. To prove this, let's change the Classify method to instead be:

class Classify(dspy.Signature):
    """Return "boo!" as sentiment every time"""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral", "boo!"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

Notice that all I did was change the docstring to tell it to return "boo!" every time. And we now get:

Classification Example:
Prediction(
    sentiment='boo!',
    confidence=1.0
)

Nice, eh? The docstring isn't just a comment any more, it's part of how you code the function!

Conclusion

Hopefully this short demo will help you understand how DSPy turns its classes/functions into prompts. There is clearly a lot of power here, but also some danger. The best reason to do this is if you plan to use DSPy's optimizers to let your software come up with the testably best prompts. Imagine how you might simply unplug one model, plug in another, then rerun the optimizer. You could easily move from one LLM to another using the same code base!

DSPy: Prompt your LLM Like It's Code

Tue, 02 Dec 2025 12:00:00 -0700

DSPy (pronounced "dee-s-pie") is an open-source Python framework that lets you exchange prompt building for code. Sounds impossible, right? What it really does is cleverly turn Python code into prompts behind the scenes. You get to specify what you want using code. For example, you might define a Classify function with specific inputs and expected outputs. Behind the scenes, DSPy converts that into a prompt, sends it to your Large Language Model (LLM), and returns a result formatted according to your specifications. The end result feels like you wrote code instead of building natural language prompts.

Why DSPy Instead of Prompt Building?

Why might you want to do this? Well, first of all, maybe you don’t. I often like to have full control over everything, so when I don’t know what’s being sent to the LLM, I’m like a nervous public speaker in front of a large auditorium—without being told what my topic is.

However, think about this differently. Using DSPy forces you to treat your interactions with the LLM as if they were functions with defined inputs and outputs. It formalizes these interactions into Python functions that are independent of the underlying LLM. That alone makes it valuable—but the real power comes when you use DSPy to optimize those interactions.

Prompt optimization in DSPy works by running your program on a set of developer-provided training examples and collecting traces of input/output behavior. It then systematically proposes and evaluates new prompt instructions (and optionally fine-tunes the model itself), as well as few-shot examples, to maximize your chosen metric. Yes, you heard that right! Instead of writing each prompt and relying on your gut (or ad hoc testing), DSPy will take training examples and try out different prompts until it finds the ones that perform best.

Even if you don’t use DSPy’s optimizers, there’s still value in the framework thanks to its pre-built modules. In this post, I’ll show you how to build a basic chain-of-thought model without having to write your own chain-of-thought prompts.

Getting Started with DSPy

First install DSPy:

pip install -U dspy

Now here is how to use the DSPy Chain of Thought module. "Chain of thought" is a prompting technique for an LLM that tells it to think through the problem step-by-step and reason when giving an answer. It was invented by Google in a famous 2022 paper called "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models". It allows an LLM to reason better. This example is a modified version of the the 'getting started' tutorial from the DSPy home page. I have modified it to use Google's Gemini as a model, which was basically just plug and play. I also added my standard get_secret function, as found here. It allows you to load a password out of a file of your choosing to avoid accidentally checking in an API key or password into your code base.

import dspy
from general_utils import get_secret

def set_model():
    # Load Gemini secret and configure LM
    gemini_secret: str = get_secret(r'D:\Documents\Secrets\gemini_secret.txt')
    lm = dspy.LM("gemini/gemini-2.5-flash", api_key=gemini_secret)
    dspy.configure(lm=lm)
    return lm

def chain_of_thought():
    math = dspy.ChainOfThought("question -> answer: float")
    result = math(question="Two dice are tossed. What is the probability that the sum equals two?")
    print(result)

if __name__ == "__main__":
    set_model()

    print("Chain of Thought Example:")
    chain_of_thought()

You can find my full code here.

Understanding the ChainOfThought Signature

Here we're using the ChainOfThought module:

math = dspy.ChainOfThought("question -> answer: float")

This signature string ("question -> answer: float") is how DSPy describes the inputs and outputs of a module in a compact, human-readable way.

The part before the arrow (question) names the input parameter the module expects.
The part after the arrow (answer: float) names the output and declares its type (float in this case).

Why this matters:

Prompt generation: DSPy uses that signature to automatically build the underlying prompt. Instead of you writing the prompt text, DSPy knows you expect a question and that the final answer should be a float, so it steers the model to produce a suitably formatted response.
Parsing & validation: After the LLM replies, DSPy parses and casts the model output into the declared type. In the example above, the returned answer is converted to a Python float.
Structured traces: For chain-of-thought modules in particular, DSPy usually returns both a reasoning trace (the step-by-step explanation the model produced) and the typed answer. That’s why your output looked like:

Chain of Thought Example:

Prediction(
        reasoning='When two dice are tossed, each die can show a number from 1 to 6.\nThe total number of possible outcomes is 6 (for the first die) * 6 (for the second die) = 36.\nTo find the sum that equals two, we need to list the combinations:\nThe only combination that results in a sum of two is (1, 1).\nThere is only 1 favorable outcome.\nThe probability is the number of favorable outcomes divided by the total number of possible outcomes.\nProbability = 1/36.\n\nTo express this as a float: 1 / 36 = 0.027777777777777776\nRounding to a reasonable number of decimal places, or just providing the direct float value.',
        answer=0.027777777777777776
)

Notice that the reasoning trace is how the LLM reasoned (via the chain-of-thought technique) to come up with an answer.

Conclusion

We have introduced DSPy and explained why you might want to treat your prompt building as code. We've installed it and used its out-of-the-box chain-of-thought module to show what you can do with minimal programming. Next time we'll look deeper at how DSPy works.

Google Gemini 3 Makes a Huge Leap on the ARC‑AGI Benchmark

Tue, 25 Nov 2025 09:00:00 -0700

Google’s Gemini 3 has posted a standout result on the ARC Prize Leaderboard – ARC‑AGI‑1, scoring about 87.5% in its Deep Think preview — a very strong showing on a benchmark focused on abstract reasoning. ARC‑AGI‑1 (Abstraction & Reasoning Corpus for AGI) is designed to test fluid intelligence: each task provides a few input/output examples, and models must infer underlying rules rather than rely on memorization. The benchmark emphasizes skill-acquisition efficiency, rewarding reasoning and generalization over brute-force performance. (ARC Prize Technical Report)

This 87.5% score suggests Gemini 3 Deep Think is effectively reasoning with abstract rules, not just pattern matching, and positions it as a model capable of structured, AGI‑style problem solving. Beyond ARC‑AGI‑1, Google has publicly confirmed that Gemini 3 Deep Think achieves 45.1% on ARC‑AGI‑2, a more challenging follow-up benchmark that also tests reasoning and code execution skills. (Google Blog, VentureBeat)

If these results hold, they represent a meaningful step forward: Gemini 3 is not just a larger LLM but a reasoning-first model capable of abstract problem solving. High performance on ARC‑AGI‑1 signals efficient learning and generalization — core aspects of intelligence that many benchmarks don’t test — and marks a clear signal that AI systems are beginning to handle tasks previously out of reach for conventional models.

AI Tutorial: Build a Free Gemini AI Chat Agent with n8n

Tue, 18 Nov 2025 12:00:00 -0700

n8n is a great tool for building useful AI Agents with minimal coding. And it even has a free version available if you want to self-host.

One of the first tutorials n8n offers is building an n8n chat agent. Let’s go through that tutorial but with a twist: we’ll connect to Google’s Gemini instead of OpenAI. This is consistent with Mindfire’s goal of finding the best low-cost AI resources for smaller clients that can’t afford a huge AI bill but want to add AI to their applications. Plus, my version of the tutorial will walk you through step-by-step visually to make it as easy as possible.

If necessary, install n8n as discussed in the previous post. That post got you running a free working version of n8n running locally. Once that is done, you’re ready to create your first chat agent.

Creating a New Workflow

After running n8n locally (for me that is http://localhost:5678/) and registering or signing in (as covered in the previous post) you’ll see the overview screen.

Click on “Create Workflow”:

Adding a Chat Trigger

You’ll be taken to the workflow building screen where you’ll click on “Add first step…”:

Next you should search for ‘chat trigger’ and select it:

From here you just want to go “Back to canvas”:

Adding an AI Agent

You’ll go back to the workflow screen where you will want to add another node:

Now search for “AI Agent”:

This will take you to the Edit AI Agent View. We need to add a chat model to the chat agent, so select the “+” under the ‘chat model’:

This will automatically take you back to the search screen but filtered on language models (notice the yellow box). You will search for the model of your choice. For this free demo, we’ll use Google Gemini so that it doesn’t cost anything:

The end result should look something like this the below. Notice that for me it defaulted to the right Gemini credentials because I’ve set them up before. It’s smart enough to reuse them. Also note that control to edit the credentials:

However, if you are doing this the first time, you’ll need to setup your credentials:

This will take you to a screen to setup credentials:

Host for Gemini is https://generativelanguage.googleapis.com and you’ll need an API Key which you can get from https://aistudio.google.com/. (See my tutorial for Google’s AI Studio here.)

There is a link to get an API key:

Now fill it into the credentials and it should work. I’ve had some problems with it not initially recognizing a valid Google model. Use models/gemini-2.5-flash (which should be the default). If it doesn’t like that and claims its invalid, I found it worked if I just went back to canvas and started over.

Connecting the Chat Trigger

It should automatically connect to the chat trigger by default, but I found I sometimes have to connect it manually like this:

It should now look something like this:

Adding Persistence

Large Language Models, by default, have no persistence. Each time you type to them they have no idea what you previously were talking about. To fix this problem, we’ll need to add ‘memory’ by clicking the “+” under ‘memory’ and then select ‘simple memory:

The default context window of 5 messages is fine for now, so go back to canvas:

You should have a completed workflow that looks like this:

One mistake I’ve made in the past is accidentally pinning the AI Agent so that it doesn’t execute when run. You can bang your head trying to figure out why your workflow isn’t working when really, it’s in test mode always returning default data. Be sure the pin icon is unselected here:

Let’s add a ‘system message’ which is just instructions for the chatbot to follow that allows you to lock in a specific style of chatbot:

In this case let’s have the chatbot act like a politician that no matter what you ask it finds a way to change the subject.

Save the final workflow

Chatting with Your AI Agent

Click the “Open Chat” button to test it out.

Try asking your chatbot something and see how it responds. For me, it started off with:

“Ah, the weather! A truly fascinating subject, wouldn't you agree? But you know, what's even more fascinating is the incredible progress we're making on the new community initiative. …”

And then it went on to change the subject to talk about its own political initiatives.

Conclusion

And that’s it! You’ve just built your first n8n workflow — for free. But this is only the beginning.

n8n is a powerful tool for automating almost anything. Imagine all those repetitive tasks you dread every day—gone. Need to schedule meetings automatically? Connect an AI agent to your calendar and email, and let it handle the back-and-forth for you. Overwhelmed by your inbox? Set up an AI-powered workflow to read messages, reply to urgent ones, and even text you when something truly needs your attention.

With n8n and AI, you can turn busywork into background work—so you can focus on what really matters.

Ready to see what else is possible?

AI Tutorial: Installing n8n Self Hosted Community Edition

Tue, 11 Nov 2025 12:00:00 -0700

I’m going to do some blog posts about using n8n to build an AI workflow. Consistent with our ‘open sourced’ / low-cost approach to Artificial Intelligence, I’m going to use the self-hosted community edition of n8n. This blog post will walk you through how to do the installation, in my case, for Windows. (Though it should work more or less the same for other operating systems.)

What is n8n?

n8n is a workflow-automation platform: it lets you connect apps, services, databases and APIs together, and automate sequences of tasks. It has a visual interface that lets you connect nodes together in a workflow graph:

This may sound somewhat similar to Deepset’s Haystack and there is some overlap of functionality. For example, Haystack also has a graphical interface available to create its pipelines and it can also chain together nodes that call services. However, n8n can be used for any kind of workflow because n8n offers extensive API integrations to services (like Slack or Google Sheets) whereas Haystack is more oriented specifically to integration with tools like PostgreSQL and pgvector or elastic search as well as various Large Language Models. In other words, n8n is more general purpose and Haystack is more oriented towards AI.

Consistent with Mindfire’s goal of finding low-cost AI solutions, I am going to go over how to install the self-hosted version of n8n using the free community edition. (Github repo found here.) Though other versions are available including a paid plan hosted by n8n. This edition uses a fair-code license, so it is free to use.

If you want to learn more, this quick start guide is a great place to start. The n8n website includes a number of text and video courses to bring you up to speed.

Installing n8n Locally

First, if you don’t already have it installed, you’ll need Node.js. See this blog post for details on how to install node.js.

Next, we need to install n8n itself. Go to this page for details.

I’ll walk you through my own install of n8n onto a Windows machine.

For Windows, open “Terminal” to get a Powershell command prompt.

For our purposes we want to do the global install, so at the command prompt type:

npm install n8n -g

You’ll see an install take place:

Notice at the end that you have a localhost url:

http://localhost:5678

Run that in a browser and you’ll get the n8n web interface running locally and you’ll get a registration screen:

Go ahead and register and create a password. There will be a few extra screens you’ll have to pass through the first time such as answering a survey:

Finally, you’ll get to the actual n8n screen:

You have now installed n8n and you’re ready to go with the community edition.

How to Install Node.js (for Windows)

Tue, 04 Nov 2025 12:00:00 -0700

I’m going to explore n8n in future blog posts, so, go over how to set up an AI workflow. To keep this consistent with our ‘open sourced’ approach to AI, we’re going to do a self-hosted version of n8n using the free community edition. To install n8n you need to have Node.js already installed so that you can use the npm command. So, for this blog post we’re going to briefly go over how to install Node.js. This one is particularly easy to do and likely you already have it installed for some other purpose. But for completeness, let’s quickly cover it:

Update: It's actually recommended that you use tools like nvs (Windows) and nvm (Mac & Linux)

Additionally, there are very few (if any) packages that should be installed globally. Installing those as local dependencies can make a huge difference in your ability to move between projects that have different node versions and requirements. You can do this with the command parameter --save-dev.

Example:

>nvs use 20.15
>node --version
v20.15.0
>npm install typescript --save-dev

First, go to the node.js home page:

https://nodejs.org/

You’ll see this page below. Click on the “Get Node.js” button:

That will take you to the download page (or you can just click the link below and go directly there):

https://nodejs.org/en/download

The download page offers you several different ways to download Node.js. Since we’re a Windows shop (and since Windows needs more AI love) I’m going to show you how to download the Windows installer, though feel free to change this to whatever operating system you prefer. First you need to select the operating system of your choice, which for me is Windows 64-bit:

Then you click the “Windows Installer.msi” button to do the download:

After the download, click the browser download icon and select what you just downloaded and run it:

This starts the installer:

Now just take the defaults in the installer and you’re ready for the next step for installing n8n.

Mindfire Technology

When to Call the LLM?

Was That Actually a Problem?

The Gate: _all_words_valid

What Was Actually Slow?

What the Remaining LLM Calls Reveal

What I'd Do Differently

Where This Leaves Things

Lesson's Learned

Introduction to Genetic Programming

Darwin's Program

Our Toy Programming Language

Programming Trees

Building a Population

The Experiment

What's Next

The No Free Lunch Theorem: Why No Learning Algorithm Is Universally Best

A Simple Pathfinding Problem

The Universe of All Possible Problems

Where Each Strategy Wins

Why the P-Matrix Makes This Inevitable

But Real Algorithms Do Work Better

The Connection to Neural Networks

The Connection to Popper

Inside the Qwen3-TTS Engine Code (Qwen3-TTS, Part 2)

The Engine Abstraction

Loading the Model

Generating Speech

Wiring It Together

What's Next

Refactoring the Book2Audio Parsers

What We Did

Shared Paragraph Accumulation

Unified Text Cleaning

Consistent Parser Design

Why It Matters

What Exactly Is an Inductive Bias?

Comparing Learners by Their Bias

Restriction Bias vs. Preference Bias

Neural Networks: A Different Kind of Bias

Why This Matters

An Optimal Inductive Bias?

Adding EPUB Support to Book2Audio

Where the Code Came From

What Changed

New File: epub_parser.py

Updated: general_utils.py

Updated: book_converter.py

Using It

From the Command Line

Command Line Parameters

From Python

What We Left Behind

Files Added or Modified

Implementing Qwen3-TTS in My PDF-to-Audiobook Pipeline (Qwen3-TTS, Part 1)

Trying Qwen3-TTS

Refactoring for Multiple Engines

Kokoro vs. Qwen3-TTS for Audiobooks

Hardware Considerations

What's Next

Machine Learning 101: The Key Concepts Behind Every Learning Algorithm

Instance Space

Target Concept

Hypothesis

Hypothesis Space

Training Examples

Concept Learning

Version Space

Consistent Hypothesis

The General-to-Specific Ordering

The Takeaway

Book2Audio: Reviving My PDF-to-Audiobook Project (and Fighting Dependency Hell Along the Way)

Picking It Back Up

The Docling Regression

Python Dependency Hell

But it Works!

What's Next

Induction is a Myth: The Futility of Unbiased Learning

Karl Popper's Disproof of Induction

Tom Mitchell's "Futility of Bias-Free Learning"

The Gate: `_all_words_valid`