Mindfire Technology

Adventures in LangChain's "Quick Start Tutorial" (Using Ollama)

Fri, 20 Feb 2026 12:00:00 -0700

Suppose you want to learn LangChain, so naturally you go to their quick start tutorial page. And here is what you find as their first suggested tutorial:

from langchain.agents import create_agent

def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

agent = create_agent(
    model="claude-sonnet-4-5-20250929",
    tools=[get_weather],
    system_prompt="You are a helpful assistant",
)

# Run the agent
agent.invoke(
    {"messages": [{"role": "user", "content": "what is the weather in sf"}]}
)

Okay, but this requires you to run Claude. But doesn't Claude require an api key? Well, it doesn't say anything about that. Maybe there's a free Claude api tier that requires no key?

Could not resolve authentication method. Expected either api_key or auth_token to be set.

Oops, nope. You need a Claude api key. I guess they were assuming I'd just know that and already have it set in my environment? Seems like a weird assumption for a quick start tutorial, but okay, I guess?

No worries, I'll just go get a Claud api key.

But wait, doesn't Claude require me to pay for tokens?

Oh, whew! Claude has a free tier for testing! Good! Okay, not too bad. Let's try that quick start tutorial again.

Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits.

Arg! Okay, apparently Claude is lying and there is no free tier! Even though I took that verbiage directly off the API key page for Claude, I guess they meant I only had access to the web interface?

I tried asking the LangChain AI why this didn't work and it recommended I try out the OpenAI free tier instead and even helpfully gave me a rewrite to the 'quick start tutorial' that would do this.

Wait, doesn't OpenAI have no free tier? Didn't they sunset that like two seconds after they went live and ended up with so many users? But surely the LangChain AI knows what it's talking about, right?

You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/...

Nope, the LangChain AI doesn't have a clue.

Okay, no worries, I'll just rewrite this 'quick start tutorial' to use ollama. Here is what I try:

# pip pip install -U langchain langchain-ollama
# Requires Python 3.10+
# pip install -U langchain langchain-ollama
# Requires Ollama installed + "ollama pull llama3.2"
from langchain_ollama import ChatOllama  # Changed from OpenAI
from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

# No secrets needed!
llm = ChatOllama(model="llama3.2:1b")  # Free local model
agent = create_agent(
    model=llm,
    tools=[get_weather],
    system_prompt="You are a helpful assistant.",
)

# Run agent
# Our message
message = {
    "messages": [{"role": "user", "content": "what is the weather in sf"}]
}

result = agent.invoke(message)

# Print full result (shows all messages + tool calls)
print("Full result:", result)

# Print JUST the model's final response
print("Model response:", result["messages"][-1].content)

Surely, I'm finally ready to use this 'quick start tutorial', right?

Model response: There is no weather data for that city.

What!? Okay, I was using Llama 1b. That sad little model probably can't figure out how to use the tool calling in this 'quick start' tutorial.

Let's try Lama3.2:3b instead. I'll also improve the system prompt and get a lot more specific.

Current weather conditions in San Francisco are not available.

It must not be using the provided tool. Let me just set a breakpoint.

Nope, it's using the tool...

Let's just run it again, just to see what happens:

Model response: I made a mistake! The actual response from the tool is:

"Currently Sunny
Temperature: 62°F
Conditions: Clear
Wind: Light (5 mph)
Sky Conditions: Partly Cloudy"

So, to correct my previous response: The weather in SF is currently sunny with a temperature of 62°F.

Wow, it got the sunny right, finally, but it basically just made the rest of that answer up entirely. Sigh.

Well, there you go. Here's my final revised 'quick start' code to use LangChain with oLlama. Good luck! (You'll need it.)

# Requires Python 3.10+
# pip install -U langchain langchain-ollama
# Requires Ollama installed + "ollama pull llama3.2"
from langchain_ollama import ChatOllama  # Changed from OpenAI
from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"

# No secrets needed!
llm = ChatOllama(model="llama3.2:3b")  # Free local model
agent = create_agent(
    model=llm,
    tools=[get_weather],
    system_prompt="""
    You are a weather assistant.
    1. ALWAYS use get_weather tool for weather questions
    2. REPORT EXACTLY what the tool returns - do not make up data
    3. Tool result = actual weather data
    4. Base your answer ONLY on tool output
    """,
)

# Run agent
# Our message
message = {
    "messages": [{"role": "user", "content": "what is the weather in sf"}]
}

result = agent.invoke(message)

# Print full result (shows all messages + tool calls)
print("Full result:", result)

# Print JUST the model's final response
print("Model response:", result["messages"][-1].content)

Github repo found here.

If you need help with your Artificial Intelligence solutions, we're here to help.

Weekend Warrior Project: AI Patent Checking

Fri, 06 Feb 2026 12:00:00 -0700

Ever had an idea and wondered, “Wait… has someone already patented this?” At Mindfire Tech, we recently faced that exact problem. We were working on an AI patent, and before we could go any further, we needed to know if the concept was already patented.

Sure, we could have spent hours — maybe days — digging through USPTO records, manually reading abstracts, and trying to make sense of hundreds of documents. Or… we could let AI do it for us.

One weekend later, we had a fully functioning AI Patent Search agent. In a few hours, we built a tool where you just paste in your patent abstract (or the full patent!), and it will:

Generate intelligent search queries.
Search the USPTO patent database automatically.
Refine queries if there are too many or too few results.
Retrieve abstracts and documents.
Rank the results by relevance.

All you have to do is review the final top results — the agent does the rest. The funniest part? Writing the AI agent was far easier than trying to do this work manually. Instead of getting lost in endless PDFs and XML files, we spent the weekend building a tool that could save weeks of research for anyone working on patents.

It’s the kind of “weekend warrior” project that feels like magic: a fully functional AI agent built in a single weekend, solving a problem that used to take days. At Mindfire, that’s exactly the kind of creative, high-impact AI work we love doing for our clients.

How It Works: The Tech Behind the Magic

While the AI agent feels seamless, there’s a lot going on under the hood. Here’s a breakdown of the technologies that make this possible:

1. Gradio for a Clean, Instant UI

We used Gradio, a Python library, to create a web interface in minutes.

Textbox for your patent idea
Dropdown to select an AI model
Button to trigger the search
Expandable previews of retrieved patents

All interactive, all local — no frontend frameworks, no headaches.

2. Local LLMs with Ollama

All AI reasoning happens locally using Ollama.

Models like Gemma, Llama, and Mistral run right on your machine
No data leaves your system — perfect for confidential IP
Fast enough to generate search queries and semantic rankings in seconds

We used Ollama as the backbone for our AI agent, orchestrated with DSPy.

3. DSPy: Structured AI Orchestration

DSPy provides the “brains” of the agent:

Generates structured queries from your patent idea
Refines queries if the first pass fails
Scores patents for relevance using their abstracts

DSPy enforces strict input/output formats, which keeps the AI’s predictions consistent and reliable. It’s what turns a raw LLM into a real research assistant.

4. Multi-Stage Patent Retrieval

The agent searches the USPTO using our Python wrapper for their API:

Submits structured keyword queries
Filters only granted patents
Fetches up to hundreds of results per query

If a query returns zero or too many results, the agent automatically refines it.

5. FlashRank Reranking

Once we have raw results, we use FlashRank, a fast CPU-based reranker:

Scores patents by relevance to your idea
Works without GPUs or PyTorch
Reduces hundreds of results to the most relevant top candidates

This ensures you only see what really matters.

6. Full Patent Document Retrieval

For the top patents, the agent fetches the full XML from the USPTO:

Extracts abstracts with lxml
Prepares server-side previews for your review
Keeps everything local and ready for the next step

No more opening PDFs one by one — the agent handles the tedious part.

7. Semantic Reranking with LLMs

Finally, the agent uses the LLM again to read and rank abstracts semantically:

Scores each abstract 0–1 for relevance
Sorts patents so the top few are the ones you actually care about
Provides a concise, actionable list of results, including the actual abstracts.

This makes the final output feel like a real research assistant — not just a search engine.

The Result

In a single weekend, we created an agent that:

Understands your patent idea
Queries official government databases
Retrieves, parses, and ranks hundreds of patents
Presents a clean, interactive review interface

It’s fast, reliable, and ready to save anyone hours of manual research.

AI Patent Search Agent Pipeline

The diagram above shows how the agent flows from idea input to semantic scoring, highlighting all the key stages of retrieval, refinement, and ranking.

Why Mindfire Can Help

Building an AI prototype in a weekend is fun — but imagine what our team can do with more time and your unique business challenges.

At Mindfire Tech, we specialize in:

Custom AI agents for research and analysis
Automation that saves weeks of manual work
Secure, locally hosted AI solutions for sensitive projects

If you’re curious about how AI can transform your workflows — patent research or beyond — we’d love to talk.

After all, if we can build a full patent search agent in a weekend, imagine what we could build for your business.

From Prompts to Applications: A Beginner’s Introduction to LangChain

Fri, 30 Jan 2026 12:00:00 -0700

If you’ve spent any time experimenting with large language models, you’ve probably had this experience: the first few prompts feel magical — and then things get messy and difficult. You want your model to look things up, remember context, call tools, or follow a multi-step process. Suddenly, a single prompt isn’t enough. And those dang Large Language Models (LLMs) seem to have a mind of their own.

That’s the gap LangChain was created to fill.

LangChain is an open-source framework designed to help developers move from isolated LLM calls to structured AI applications. Instead of treating a language model as a black box that simply returns text, LangChain encourages you to think in terms of workflows — sequences of steps that combine models, tools, and data sources into something more reliable and reusable. This philosophy is central to how the project describes itself and why it exists (LangChain Philosophy).

What LangChain Actually Is

At a high level, LangChain provides abstractions for working with language models in a consistent way. It standardizes how you connect to different model providers and how models interact with external systems. The core idea is simple but powerful: Models should be used for more than just text generation - they should also be used to orchestrate more complex flows that interact with other data. (See LangChain Philosophy).

This shows up in LangChain’s building blocks. You’ll often hear terms like chains, tools, and agents. Chains represent ordered steps of computation, tools allow models to interact with external data or APIs, and agents enable models to decide dynamically which tools to use. Together, these concepts make it easier to build things like retrieval-augmented generation (RAG), multi-step reasoning pipelines, and AI assistants that can interact with real-world data.

Installing LangChain

LangChain is intentionally easy to get started with, especially for Python developers. Installation typically starts with the core package, followed by optional integrations for the model providers you plan to use:

pip install -U langchain

Or do a provider specific version like this:

pip install -U langchain-openai

Or (better yet for us low cost AI solution types) this:

pip install -U langchain langchain-ollama

This modular approach lets developers start small and only pull in what they need, while still leaving room to grow into more advanced use cases later (see the official LangChain Installation Docs).

A Quick Start Example

This short quick start tutorial should get you started:

from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage

# Initialize the Ollama model
llm = ChatOllama(model="llama3.2:1b")

# Create a human message
message = HumanMessage(content="Write a short introduction about LangChain")

# Generate a response
response = llm.generate([[message]])

# The response object contains generations
print(response.generations[0][0].text)

You will have to have ollama already installed and llama3.2:1b downloaded.

When LangChain Shines

LangChain works best when your application needs more than a single prompt and response. If you’re building workflows that involve retrieving information from documents, calling external APIs, coordinating multiple LLM calls, or guiding a model through a multi-step reasoning process, LangChain provides helpful structure without forcing a rigid architecture.

It is especially popular as a prototyping and experimentation tool. Many developers use LangChain to explore ideas like agent-based workflows or retrieval-augmented generation (RAG) systems before deciding how much structure they want to carry forward into production. Examples like agentic GraphRAG systems built with LangChain show how it can connect models, tools, and data sources in flexible ways (Agentic GraphRAG with LangChain).

When to Be Careful

Despite its popularity, LangChain is not universally considered production-ready out of the box. Some developers argue that its abstractions can introduce unnecessary complexity, make debugging harder, or obscure performance characteristics in real-world systems. This concern comes up frequently in discussions about production RAG pipelines, where fine-grained control over retrieval logic, latency, and observability is often critical (LangChain Is Not for Production Use — Here Is Why).

This doesn’t mean LangChain shouldn’t be used at all — but it does mean it should be treated as a toolkit rather than a complete solution. Strong engineering practices are still required to turn prototypes into reliable, maintainable applications.

LangChain in the Broader Ecosystem

LangChain exists alongside several other frameworks with overlapping goals. Tools like Haystack focus more heavily on search and retrieval performance, while newer projects like LangGraph emphasize lower-level control over agent workflows. Which framework makes sense depends largely on your use case, performance requirements, and tolerance for abstraction (LangChain vs Haystack, LangGraph vs LangChain).

Final Thoughts

LangChain is best understood as a bridge. It connects the early excitement of prompt-based experimentation with the practical reality of building real AI applications. For beginners, it offers a way to think beyond prompts and start designing systems composed of models, tools, and workflows. Used thoughtfully, LangChain can be an effective stepping stone from experimentation to production — as long as its tradeoffs are clearly understood.

AI: Bubble or Breakthrough? A Look at the Risks and the Rewards

Tue, 20 Jan 2026 12:00:00 -0700

Artificial intelligence is arguably the most hyped technology in decades. But hype isn’t the same as reality. In the debate over whether AI is in a bubble, there is both a pessimistic and a more optimistic view. Cory Doctorow warns that the economic foundations of AI are shaky and potentially disastrous, and Jeff Bezos, who agrees there’s a bubble — but insists it’s the good kind and heralds long-term benefits.

Let’s explore both sides with concrete examples.

The Pessimistic View: AI as “Funny Money” and Fragile Economics

Cory Doctorow’s article “The real (economic) AI apocalypse is nigh” argues that today’s AI bubble isn’t built on sound economics but on massive investor excitement and weak financial fundamentals. He paints a picture of an industry where capital flows in without sustainable revenue models or profit paths — classic signs of a bubble that could end badly.

Doctorow argues many AI firms “keep the lights on by soaking up hundreds of billions of dollars in other people’s money and then lighting it on fire.” He argues it’s a dependency on the continual infusion of new capital, hardly a sustainable business model.

Doctorow points out that, unlike past emerging technologies, each generation of AI technology costs more to build and operate than the last, with little evidence that those costs decline or that scaling will improve profitability.

He also points out that some AI data center companies are reportedly collateralizing loans with tens of thousands of GPUs, despite these chips losing value quickly — a bizarre financial setup that signals desperation rather than strength.

Even scarier is where Doctorow highlights practices where the same money is booked as an investment by one company and revenue by another — such as Microsoft “investing” in OpenAI by providing server access, then counting that as revenue. That’s not real earnings; it’s accounting sleight-of-hand.

Finally, he quotes a venture capitalist suggesting AI firms would need to sell hundreds of billions worth of services just to break even, a goal so large it’s hard to take seriously given current revenue levels. Is there really this much demand for AI in real life?

Doctorow's view is that this whole thing is just hype run out of control, leading to bad decisions—such as firing people over AI that can't do the job. In Doctorow's opinion, this is just the market gone mad.

The Optimistic Take: Jeff Bezos on the “Industrial Bubble”

But not everyone sees this as pure madness. Jeff Bezos — longtime tech leader and founder of Amazon — also acknowledges we’re in bubble territory, but his perspective is more nuanced.

Bezos said at Italian Tech Week 2025 that AI’s rapid growth and inflated valuations fit the classic pattern of a bubble, where “every experiment and every company gets funded — the good ideas and the bad ones alike.”

Bezos compared the AI bubble to past tech booms particularly the dot-com bubble. During the dot-com bust, Amazon’s stock famously plummeted from roughly $113 to around $6 a share during the early 2000s bust, even as its business fundamentals were improving. This shows how markets can wildly misprice companies in a bubble.

Bezos also points out that during the dot-com bubble everyone was chasing the Internet. Fiber-optic companies went bankrupt laying fiber, but the fiber they laid was bought out by other companies and is still around. The overall benefit to society was real.

This actually makes sense. Everyone can sense that AI is a transformative technology just like the Internet was. Back then everyone was trying to figure out how to use (and preferably dominate) the Internet because they knew there was massive profits on the line. So, from an investor's perspective, it made sense to chase these profits even if you risked losing everything. And for the companies that won this mad dash—Amazon, Google, Netflix, Facebook, etc—the rewards were well worth the risk.

Notice how this doesn't deny the problems Doctorow highlights, but it explains it differently. Instead of trying to explain it as market madness, it explains it as a correct understanding that huge profits are on the line. The problem is that no one knows at this point who the big winners are going to be.

This is what Bezos calls an industrial bubble: capital may flow to losers, but the underlying technology creates lasting value. Compare this to the bad kind of bubble, such as the 2008 housing bubble. That bubble was all bad because it really was just market madness with on actual paradigm-changing technology at its center.

Bezos points out that during hype cycles, investors often can’t distinguish good ideas from bad ones — meaning capital pours into projects that may never have viable products. But in the process, real innovation also gets funded.

Importantly, Bezos argues that just because valuations are frothy doesn’t mean the technology itself isn’t transformative. The dot-com bubble was a bubble, but the Internet really did transform everything. So, Bezos maintains that AI will change every industry and that the benefits to society will be “gigantic.”

So What Should We Make of It?

🚨 The Bubble Side

AI has massive investment without clear returns.
Financial engineering and accounting quirks mask true economics.
If capital stops flowing, many companies might fail spectacularly.

🌟 The Breakthrough Side

Innovation doesn’t happen without experimentation and funding, even if it’s wasteful at times.
Past industrial bubbles (Internet) left real infrastructure and products behind.
AI’s underlying technical progress isn’t a mirage — it’s real and pushing boundaries.

So I Should Be Afraid of the AI Bubble?

Surprisingly, no — though, you probably should be concerned. Take a look at this chart of the Shiller PE ratio and our three bubbles:

A Shiller PE is a price-to-earnings ratio, except smoothed out over 10 years. It's a great way to fundamentally value the market historically that smooths out all the local ups and downs. During the dot-com bubble the Shiller PE got up to 44 against a historical average of closer to 20. We are currently at 40. Even the 2008 housing bubble only got up to 28. So there is a lot of potential downside in this AI bubble. The stock market is clearly overvalued in large part due to the AI bubble.

Conclusion: Bubble or Breakthrough?

The answer might be both.

AI today shows many signs of a bubble: unsustainable valuations, weak unit economics, and overfunded startups with no path to profit. That’s worth worry — especially for investors and workers whose livelihoods depend on sound business fundamentals.

But if we look at history, bubbles aren’t uniformly bad. As Jeff Bezos emphasizes, when the dust settles, the infrastructure and breakthroughs left behind can help reshape industries for decades.

Whether you come down on the doom side or the long-term value side, one thing is clear: AI’s story is just beginning, and its economic impact will be debated for years to come.

In the meantime, one value that can be certain from AI is its use in projects and software that have an immediate purpose and measurable benefit or profitability. Mindfire Tech is committed to providing state-of-the-art software solutions and we believe AI can be a great boon to those who know how to use it properly. If you are interested in learning more about how AI could be used in your company or in your future projects, please don't hesitate to reach out for a free consultation.

Sources:
- Cory Doctorow, The real (economic) AI apocalypse is nigh — Medium. :contentReference[oaicite:18]{index=18}
- Jeff Bezos on the AI bubble, Italian Tech Week 2025 — various reporting. :contentReference[oaicite:19]{index=19}

Building a Local DeepSeek R1 Chatbot with Chainlit

Tue, 06 Jan 2026 12:00:00 -0700

In the previous post we used Streamlit and Ollama to build a local Deepseek R1 chatbot. Let's now do the same thing using Chainlit as our UI to try out Chainlit. Chainlit provides an elegant, real-time chat interface out of the box, and it works beautifully with models running through Ollama.

Just like before, DeepSeek R1 will stream both its thinking process and its final answer. Chainlit is dedicated specifically to building chatbots and doesn't seem quite as flexible as Streamlit. This led to at least one problem I'll explain below.

The full working code is included in this post and in my GitHub repo.

What You'll Need

Python 3.11+ installed
Ollama installed (see this post)
DeepSeek R1 pulled locally:
ollama pull deepseek-r1:1.5b
Chainlit:
pip install chainlit
Ollama Python library:
pip install ollama

The Complete Code

Below is the full app.py file we'll walk through:

import chainlit as cl
import ollama


def convert_latex_delimiters(text):
    """Convert LaTeX delimiters from backslash-bracket to dollar signs"""
    if not text:
        return text
    # Replace display math delimiters
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    # Replace inline math delimiters
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')
    return text


@cl.on_message
async def on_message(message: cl.Message):
    """
    Handles incoming messages from the user, sends them to the LLM,
    and streams the response back to the Chainlit interface with thinking process.
    """
    # System prompt for the AI
    system_message = {
        "role": "system",
        "content": """You are an advanced AI assistant powered by the deepseek-r1 model.

Guidelines:
- If you're uncertain about something, acknowledge it rather than making up information
- Format your responses with markdown when it improves readability
"""
    }

    # Create a step for thinking and messages for streaming
    thinking_step = cl.Step(name="💭 Thinking", type="tool")
    final_answer = cl.Message(content="")

    accumulated_thinking = ""
    accumulated_answer = ""

    try:
        # Request completion from the model with streaming and thinking enabled
        stream = ollama.chat(
            model="deepseek-r1:1.5b",
            messages=[
                system_message,
                {"role": "user", "content": message.content}
            ],
            stream=True,
            think=True,  # This is the critical parameter for Ollama native API
        )

        thinking_started = False
        answer_started = False
        answer_buffer = ""

        # Stream the response to the UI
        for chunk in stream:
            chunk_msg = chunk.get("message", {})

            # Handle thinking content
            if chunk_msg.get("thinking"):
                if not thinking_started:
                    thinking_started = True
                    await thinking_step.send()

                thinking_text = chunk_msg["thinking"]
                accumulated_thinking += thinking_text
                thinking_step.output = convert_latex_delimiters(accumulated_thinking)
                await thinking_step.update()

            # Handle answer content
            if chunk_msg.get("content"):
                if not answer_started:
                    answer_started = True
                    if thinking_started:
                        # Finalize the thinking step
                        await thinking_step.update()
                    await final_answer.send()

                answer_text = chunk_msg["content"]
                accumulated_answer += answer_text
                answer_buffer += answer_text

                # Only update every 10 characters or so to avoid overwhelming the socket
                if len(answer_buffer) >= 10:
                    await final_answer.stream_token(answer_buffer)
                    answer_buffer = ""

        # Send any remaining buffered content
        if answer_buffer:
            await final_answer.stream_token(answer_buffer)

        # Update final answer with LaTeX conversion
        final_answer.content = convert_latex_delimiters(accumulated_answer)
        await final_answer.update()

    except Exception as e:
        error_msg = cl.Message(content=f"❌ Error generating response: {str(e)}")
        await error_msg.send()


@cl.on_chat_start
async def start():
    """
    Sends a welcome message when the chat starts.
    """
    await cl.Message(
        content="👋 Hello! I'm powered by **DeepSeek R1**. I'll show you my thinking process before answering.\n\n"
                "Try asking me a math problem or reasoning question!"
    ).send()

Breaking Down the Code

Chainlit Event Hooks

Chainlit uses decorators such as:

@cl.on_message
@cl.on_chat_start

These act as event listeners.
- on_chat_start fires once when the UI loads.
- on_message fires every time the user sends a message.

LaTeX Conversion Helper

DeepSeek R1 frequently uses LaTeX but with delimiters that Chainlit doesn't render by default. We fix that:

def convert_latex_delimiters(text):
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')

Same logic as the Streamlit version, just applied before displaying anything.

Thinking Step

Chainlit allows you to create "steps" that appear in the UI:

thinking_step = cl.Step(name="💭 Thinking", type="tool")

As the model streams reasoning content, we update this step live. The user can watch DeepSeek R1 deliberate.

Unfortunately, I couldn't figure out (in time for this post) how to get the thinking to sit on top of the answer. So if you are watching it think the spot where the final answer goes scrolls off the top of the screen. I tried several ideas on how to fix this and none worked. Good job Chainlit getting your app that has only one job to not do it right! (At least not easily.)

Streaming with Ollama

The heart of the system:

stream = ollama.chat(
    model="deepseek-r1:1.5b",
    messages=[...],
    stream=True,
    think=True,
)

stream=True → tokens arrive as a generator
think=True → reasoning is delivered separately from the final answer

Streaming Back to Chainlit

We listen for both thinking and final answer tokens:

if chunk_msg.get("thinking"):
    ...
if chunk_msg.get("content"):
    ...

Thinking updates go to thinking_step.update(), while answer tokens are streamed via:

await final_answer.stream_token(...)

This gives a smooth, real-time chat experience.

Welcome Message

When the chat starts, we show a friendly introduction:

await cl.Message(
    content="👋 Hello! I'm powered by **DeepSeek R1**..."
).send()

Error Handling

Any unexpected issues are passed back to the UI:

await cl.Message(content=f"❌ Error generating response: {str(e)}").send()

Running the Application

Save the file as app.py and run:

chainlit run chainlit_example.py -w

Your browser will open automatically with the Chainlit interface. Try a math or logic problem, and you'll see DeepSeek R1 stream its thoughts step-by-step before answering.

Final Result:

Key Features

Runs entirely locally—no external API calls
Chainlit displays the reasoning process as a dedicated live-updating step
Smooth streaming output
Proper math rendering
Clean and modern chat UI with minimal setup

Concerns

I never did get Chainlit to work as well as Streamlit. It tended to error out due to 'too many tokens' errors and the Latex support was hit and miss at best. I'll have to circle back on this and see if I can improve it.

Conclusion

Using Chainlit with DeepSeek R1 makes it incredibly simple to build a polished chatbot interface. With around 100 lines of code, you get live thinking visualization, Markdown rendering, history management, and a production-ready UI—all running locally via Ollama.

Feel free to adapt this example and extend it for your own experiments!

Building a Local DeepSeek R1 Chatbot with Streamlit and Ollama

Tue, 30 Dec 2025 12:00:00 -0700

With all the -- admittedly already out of date -- hype around Deepseek R1 from China, I thought it would be fun to build a simple chatbot interface for DeepSeek R1 that runs entirely on your local machine. DeepSeek R1 is a reasoning model that shows its "thinking process" before giving answers, similar to OpenAI's o1 model. We'll use Streamlit for the interface and Ollama to run the model locally.

The full code available both in this post and in my Github Repo found here.

What You'll Need

Python 3.11+ installed on your computer
Ollama installed (see this post for details)
The DeepSeek R1 model pulled in Ollama: ollama pull deepseek-r1:1.5b
Streamlit: pip install streamlit
Ollama Python library: pip install ollama

The Complete Code

Here's the full code we'll be breaking down:

# Run using: streamlit run streamlit_example.py
import streamlit as st
import ollama

st.set_page_config(page_title="DeepSeek R1 Chat", page_icon="🤖")


def convert_latex_delimiters(text):
    """Convert LaTeX delimiters from backslash-bracket to dollar signs"""
    # Replace display math delimiters
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    # Replace inline math delimiters
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')
    return text


# Initialize chat history
if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        if msg["role"] == "assistant" and msg.get("thinking"):
            with st.expander("🧠 View Thinking Process", expanded=False):
                st.markdown(convert_latex_delimiters(msg["thinking"]))
        st.markdown(convert_latex_delimiters(msg["content"]))

# Chat input
user_input = st.chat_input("Ask DeepSeek R1")

if user_input:
    # Add and display user message
    st.session_state.messages.append({"role": "user", "content": user_input})
    with st.chat_message("user"):
        st.markdown(user_input)

    # Generate and display assistant response
    with st.chat_message("assistant"):
        thinking_display = st.empty()
        answer_display = st.empty()
        accumulated_thinking = ""
        accumulated_answer = ""

        with st.spinner("Thinking...", show_time=True):
            stream = ollama.chat(
                model="deepseek-r1:1.5b",
                messages=[{"role": m["role"], "content": m["content"]}
                          for m in st.session_state.messages],
                stream=True,
                think=True,
            )

            for chunk in stream:
                chunk_msg = chunk.get("message", {})

                # Stream thinking content
                if chunk_msg.get("thinking"):
                    accumulated_thinking += chunk_msg["thinking"]
                    with thinking_display.container():
                        with st.expander("🧠 View Thinking Process", expanded=True):
                            st.markdown(convert_latex_delimiters(accumulated_thinking) + "▌")

                # Stream answer content
                if chunk_msg.get("content"):
                    accumulated_answer += chunk_msg["content"]
                    answer_display.markdown(convert_latex_delimiters(accumulated_answer) + "▌")

        # Final display without cursor
        if accumulated_thinking:
            with thinking_display.container():
                with st.expander("🧠 View Thinking Process", expanded=False):
                    st.markdown(convert_latex_delimiters(accumulated_thinking))
        answer_display.markdown(convert_latex_delimiters(accumulated_answer))

        # Save to chat history
        st.session_state.messages.append({
            "role": "assistant",
            "content": accumulated_answer,
            "thinking": accumulated_thinking or None
        })

Breaking Down the Code

Page Configuration

st.set_page_config(page_title="DeepSeek R1 Chat", page_icon="🤖")

This sets up our Streamlit page with a custom title and icon that appears in the browser tab.

LaTeX Conversion Function

def convert_latex_delimiters(text):
    text = text.replace(r'\[', '$$')
    text = text.replace(r'\]', '$$')
    text = text.replace(r'\(', '$')
    text = text.replace(r'\)', '$')
    return text

DeepSeek R1 often outputs mathematical notation using LaTeX syntax. However, it uses \[ and \] for display math and $ and $ for inline math. Streamlit's markdown renderer expects $$ and $ instead, so this function converts between the two formats.

Initializing Chat History

if "messages" not in st.session_state:
    st.session_state.messages = []

Streamlit reruns your entire script on every interaction. To preserve chat history between reruns, we store messages in st.session_state, which persists across reruns.

Displaying Chat History

for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        if msg["role"] == "assistant" and msg.get("thinking"):
            with st.expander("🧠 View Thinking Process", expanded=False):
                st.markdown(convert_latex_delimiters(msg["thinking"]))
        st.markdown(convert_latex_delimiters(msg["content"]))

This loop displays all previous messages. For assistant messages, if there's thinking content, we show it in a collapsible expander. This keeps the interface clean while still making the reasoning process available.

Handling User Input

user_input = st.chat_input("Ask DeepSeek R1")

if user_input:
    st.session_state.messages.append({"role": "user", "content": user_input})
    with st.chat_message("user"):
        st.markdown(user_input)

When a user types a message, we add it to our message history and display it immediately in the chat interface.

Streaming the Response

with st.spinner("Thinking...", show_time=True):
    stream = ollama.chat(
        model="deepseek-r1:1.5b",
        messages=[{"role": m["role"], "content": m["content"]}
                  for m in st.session_state.messages],
        stream=True,
        think=True,
    )

The critical part here is think=True. This tells Ollama to return DeepSeek R1's reasoning process separately from its final answer. The stream=True parameter makes the response appear word-by-word rather than all at once.

Processing the Stream

for chunk in stream:
    chunk_msg = chunk.get("message", {})

    # Stream thinking content
    if chunk_msg.get("thinking"):
        accumulated_thinking += chunk_msg["thinking"]
        with thinking_display.container():
            with st.expander("🧠 View Thinking Process", expanded=True):
                st.markdown(convert_latex_delimiters(accumulated_thinking) + "▌")

    # Stream answer content
    if chunk_msg.get("content"):
        accumulated_answer += chunk_msg["content"]
        answer_display.markdown(convert_latex_delimiters(accumulated_answer) + "▌")

As chunks arrive from Ollama, they contain either "thinking" or "content" fields. We accumulate both separately and display them in real-time. The "▌" character creates a blinking cursor effect showing that content is still streaming.

Finalizing the Display

if accumulated_thinking:
    with thinking_display.container():
        with st.expander("🧠 View Thinking Process", expanded=False):
            st.markdown(convert_latex_delimiters(accumulated_thinking))
answer_display.markdown(convert_latex_delimiters(accumulated_answer))

Once streaming completes, we remove the cursor and collapse the thinking expander by default. This gives users a clean final view with the option to expand and see the reasoning.

Saving to History

st.session_state.messages.append({
    "role": "assistant",
    "content": accumulated_answer,
    "thinking": accumulated_thinking or None
})

Finally, we save the complete response to our message history so it persists when the page reruns.

Running the Application

Save the code as streamlit_example.py and run:

streamlit run streamlit_example.py

Your browser will open with the chatbot interface. Try asking it a math question like "What is 15 times 23?" and you'll see it show its reasoning process before giving the final answer.

Here was my result:

One annoying aspect of Deepseek R1 is that I've seen it got stuck in a 'thinking loop'. I once asked it to give me the first 12 digits of PI and it couldn't decide if that included the '3.' or not. It kept going back and forth trying to decide for 10 minutes before I gave up. (Rather than just giving me both answers.)

Key Features

Runs completely locally with no API calls
Shows the model's reasoning process in expandable sections
Streams responses in real-time for a better user experience
Properly renders mathematical notation
Maintains conversation history

Conclusion

With just 90 lines of Python, we've created a functional chatbot interface for DeepSeek R1 that showcases one of its most interesting features: transparent reasoning. The combination of Streamlit's simple API and Ollama's local model hosting makes it easy to experiment with AI models without worrying about API costs or privacy concerns.

The complete source code is available above. Feel free to modify and extend it for your own projects!

How DSPy Optimizes Prompts

Tue, 23 Dec 2025 12:00:00 -0700

In our previous posts (here, here, and here), we looked at DSPy and how it turns Large Language Models (LLMs) into structured, type-safe Python functions. You define a class with input and output fields, and DSPy builds the prompt for you automatically.

But here’s the thing — sometimes your initial prompt isn’t perfect. The model might get the instructions a little wrong, or your few-shot examples might not be ideal. That’s where DSPy optimizers like MIPROv2 comes in. It’s a tool in DSPy that automatically tweaks your prompts and few-shot examples to make the model behave better. You can read more about it on the here.

In this post, we’ll show you how DSPy can optimize prompts using MIPROv2. We’ll go from an instruction that gives poor results to one that gives consistently correct results — and we’ll show you exactly what changes along the way. You’ll see how the model’s instructions and examples evolve, and why this makes your LLM programs more reliable.

You can find the full code for this blog post on my github repo here.

A Poorly Defined Prompt

The fact is that DSPy is so good at building prompts that I actually struggled to come up with a good simple example of how it can automatically improve prompts for you. It often got 100% accuracy on sentiment analysis right out of the box. So to give it a real challenge I rewrote my 'Classify' function like this. Originally we had:

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

And now we have the much vaguer:

class AnalyzeText(dspy.Signature):
    """Process the input."""  # ← Useless instruction
    text: str = dspy.InputField(desc="input data")
    label: Literal["Bingo!", "Hmmm...", "Oink!"] = dspy.OutputField(desc="output")  # Shuffled order!

Instead of being called "Classify" (which is a dead give away we're doing sentiment analysis) we now call it "AnalyzeText". And "sentence" and "sentiment" are now "text" and "label". Plus the labels are 'Bingo!', 'Oink!' and 'Hmmm...'

The first time I tried this, Gemini figured out on its own that 'Bingo!' was positive, 'Oink!' was negative, and 'Hmmm...' was neutral. So I had to swap orders around. And now 'Bingo!' is neutral, 'Oink!' is positive, and 'Hmmm...' is negative. So now there is no way Gemini can figure out what the correct labels are!

And sure enough, it does a terrible job labeling the data out of the box:

============================================================
BEFORE OPTIMIZATION
============================================================

📋 Initial Instruction:
"Process the input."

🎭 INVERTED Nonsense Categories (counterintuitive!):
   'Oink!'   = positive sentiment (opposite of what you'd expect!)
   'Bingo!'  = neutral/mixed sentiment (not positive!)
   'Hmmm...' = negative sentiment (not thoughtful!)

The model will naturally guess WRONG without training!

🧪 Testing on dev set (without optimization):
  ✗ 'The first half was amazing but then it fell a...' → Hmmm...  (expected: Bingo!)
  ✗ 'Not terrible but nothing special....' → Hmmm...  (expected: Bingo!)
  ✗ 'An absolute masterpiece in every way!...' → Bingo!   (expected: Oink!)
  ✓ 'I wanted to like it but it was just awful....' → Hmmm...  (expected: Hmmm...)
  ✗ 'Has its moments but overall just average....' → Hmmm...  (expected: Bingo!)

📊 Initial accuracy: 1/5 (20%)
    ↑ Should be near random chance (~33%) with no instruction!

I'm sort of surprised it even got 20% right.

Running the Optimizer

Now that we’ve seen how DSPy builds prompts automatically, let’s see how it can improve them using the MIPROv2 optimizer.

We’ll take our deliberately confusing example above to make the optimizer necessary. We also define a small training set and validation set with tricky examples to test the model. The metric simply checks whether the model predicts the expected label:

def metric(example, pred, trace=None):
    return example.label == pred.label

As we saw, this initially only gets 20% correct. The full initial prompt can also be inspected using:

lm.inspect_history(n=1)

This shows exactly how DSPy was instructing the LLM before optimization. Here is the prompt that is built initially:

System message:

Your input fields are:
1. `text` (str): input data
Your output fields are:
1. `label` (Literal['Bingo!', 'Hmmm...', 'Oink!']): output
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## text ## ]]
{text}

[[ ## label ## ]]
{label}        # note: the value you produce must exactly match (no extra characters) one of: Bingo!; Hmmm...; Oink!

[[ ## completed ## ]]
In adhering to this structure, your objective is:
        Process the input.

Let's run the MIPROv2 optimizer:

optimizer = MIPROv2(metric=metric, auto="light")
optimized = optimizer.compile(
    student=program,
    trainset=trainset,
    valset=devset,
)

...

if hasattr(optimized, 'predictor'):
    optimized_instruction = optimized.predictor.signature.__doc__
    print(f'"{optimized_instruction}"')

    original_instruction = program.predictor.signature.__doc__

After the MIPROv2 optimizer runs we get much better results:

============================================================
AFTER OPTIMIZATION
============================================================

📋 Optimized Instruction:
"You are a sentiment analysis model. Classify the given text into one of the following categories: "Oink!" for strongly positive, "Hmmm..." for negative, and "Bingo!" for mixed or neutral sentiments."

✨ INSTRUCTION CHANGED! ✨

  Before: 'Process the input.'

  After:  'You are a sentiment analysis model. Classify the given text into one of the following categories: "Oink!" for strongly positive, "Hmmm..." for negative, and "Bingo!" for mixed or neutral sentiments.'

  The optimizer learned the nonsense mapping!

🧪 Testing optimized version on dev set:
  ✓ 'The first half was amazing but then it fell a...' → Bingo!   (expected: Bingo!)
  ✓ 'Not terrible but nothing special....' → Bingo!   (expected: Bingo!)
  ✓ 'An absolute masterpiece in every way!...' → Oink!    (expected: Oink!)
  ✓ 'I wanted to like it but it was just awful....' → Hmmm...  (expected: Hmmm...)
  ✓ 'Has its moments but overall just average....' → Bingo!   (expected: Bingo!)

📊 Optimized accuracy: 5/5 (100%)
    ↑ Should be much better now!

Wow! 100% success now despite the misleading labels I'm expecting! Okay, how does it do it? What does the final prompt now look like?

System message:

Your input fields are:
1. `text` (str): input data
Your output fields are:
1. `label` (Literal['Bingo!', 'Hmmm...', 'Oink!']): output
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## text ## ]]
{text}

[[ ## label ## ]]
{label}        # note: the value you produce must exactly match (no extra characters) one of: Bingo!; Hmmm...; Oink!

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        You are a sentiment analysis model. Classify the given text into one of the following categories: "Oink!" for strongly positive, "Hmmm..." for negative, and "Bingo!" for mixed or neutral sentiments.

Whoa! It changed the instructions from "Process the input." (My default) to "You are a sentiment analysis model. Classify the given text into one of the following categories: 'Oink!' for strongly positive, 'Hmmm...' for negative, and 'Bingo!' for mixed or neutral sentiments."

No wonder it's now getting 100% correct!

Conclusion

What this example shows is the real power of DSPy’s optimizer workflow. With just a few lines of Python, we went from a deliberately confusing instruction that produced almost random results to a fully optimized prompt that got 100% accuracy on our dev set.

The key takeaway is that DSPy doesn’t just wrap an LLM in a function — it gives you control over how the model interprets your instructions and examples, and it can automatically improve them. That means:

You can use arbitrary input/output field names and even nonsense labels, and the optimizer will learn the correct mapping.

You can systematically improve your prompts without manually guessing how to rewrite them using empirical evidence from a test set.

The process is reproducible and type-safe, so you can confidently switch models or update instructions without breaking your code. If you do switch models, just re-run your optimizer and it will build prompts appropriate for the new model!

In short, DSPy + MIPROv2 turns what used to be trial-and-error prompt engineering into a structured, programmatic, and testable process. For anyone building reliable LLM-powered applications, this is a huge productivity and quality-of-results win.

How DSPy Builds Prompts

Tue, 16 Dec 2025 12:00:00 -0700

In previous posts, we talked about what DSPy, a tool you can use to treat Large Language Model (LLM) interactions as functions written in code rather than via prompting. In a follow-on post, we saw examples of how to use it out of the box.

Okay, that is all nice and good, but how exactly does it work? I mean ultimately you interact with an LLM via prompts, right? So what is all this code doing?

How Classify Works

In this post, we're going to answer that question. You can find my code for this post in my github repo. Let's start with the same Classify signature as from our previous post:

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

We then turn this into a real Classify function:

model = setup_model()
classify = dspy.Predict(Classify)

result = classify(
    sentence="This book was super fun to read, though not the last chapter."
)

Recall that the LLM produces a structured output like this:

Prediction(
    sentiment='neutral',
    confidence=0.8
)

How it Works

But how does it do it? Obviously it is building a prompt behind the scenes. Can we see what it is doing? Yes, by inspecting the history:

print("\n--- Prompt Built by DSPy ---")
print(model.inspect_history(n=1))

Where:

.inspect_history() is a method that keeps a record of every request DSPy has sent to the model in order.
The parameter n=1 means “show me the last 1 interaction.”

Here is the (somewhat long) result:

Classification Result:
Prediction(
    sentiment='neutral',
    confidence=0.8
)

--- Prompt Built by DSPy ---

[2025-11-20T10:39:12.956671]

System message:

Your input fields are:
1. `sentence` (str):
Your output fields are:
1. `sentiment` (Literal['positive', 'negative', 'neutral']): 
2. `confidence` (float):
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## sentence ## ]]
{sentence}

[[ ## sentiment ## ]]
{sentiment}        # note: the value you produce must exactly match (no extra characters) one of: positive; negative; neutral

[[ ## confidence ## ]]
{confidence}        # note: the value you produce must be a single float value

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        Classify sentiment of a given sentence.

User message:

[[ ## sentence ## ]]
This book was super fun to read, though not the last chapter.

Respond with the corresponding output fields, starting with the field `[[ ## sentiment ## ]]` (must be formatted as a valid Python Literal['positive', 'negative', 'neutral']), then `[[ ## confidence ## ]]` (must be formatted as a valid Python float), and then ending with the marker for `[[ ## completed ## ]]`.

Response:

[[ ## sentiment ## ]]
neutral

[[ ## confidence ## ]]
0.8

[[ ## completed ## ]]

What we see here illustrates how DSPy builds structured prompts:

System message: DSPy tells the model exactly what the input and output fields are, and how the response should be formatted.

Field markers ([[ ## sentence ## ]], etc.): These placeholders map your Python class fields to text in the prompt. The LLM fills them in with actual values (sentence input, sentiment and confidence output).

[[ ## sentence ## ]]
{sentence}

[[ ## sentiment ## ]]
{sentiment}        # note: the value you produce must exactly match (no extra characters) one of: positive; negative; neutral

Docstring usage: The docstring from Classify (“Classify sentiment of a given sentence.”) becomes part of the instructions, guiding the model on the task.

[[ ## completed ## ]]
In adhering to this structure, your objective is: 
        Classify sentiment of a given sentence.

This structure ensures the LLM returns a predictable, typed response — which is exactly why we get Prediction(sentiment='neutral', confidence=0.8) instead of an unstructured string.

Conclusion

In this post, we saw how DSPy automatically converts a Python class into a fully structured prompt for Gemini. By inspecting the history, we can see exactly how the class name, fields, and docstring are used to generate instructions that guide the model’s response.

This approach gives you predictable, type-safe outputs without manually writing prompts. You can experiment by tweaking input/output fields or the docstring and immediately see how the prompt — and the model’s behavior — changes.

Ultimately, DSPy lets you treat LLMs like functions in your code, while still giving full transparency into the underlying prompts. It’s a powerful way to combine the flexibility of LLMs with the reliability of typed, structured programming.

DSPy: A Powerful (But Sometimes Dangerous) Prompting Tool

Tue, 09 Dec 2025 12:00:00 -0700

In our last post, we introduced DSPy, a tool to treat prompt building for your Large Language Model (LLM) like it is a Python function. This time we're going to use another modified example off their 'getting started' page and then play with it a little and get a feel for how DSPy works.

Writing a "Classify" Function

Let's use DSPy to write a sentiment classification function. Here is the code (found in this github repo):

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

Explanation of the `Classify` function, line by line:

Let's analyze what is going on here.

class Classify(dspy.Signature):
This defines a new DSPy Signature class named Classify. By subclassing dspy.Signature, you declare that this class specifies the input/output contract for a DSPy module. (DSPy signatures docs)
"""Classify sentiment of a given sentence."""
A docstring that gives a human-readable description of the task: sentiment classification of a sentence.
sentence: str = dspy.InputField()
Declares an input field named sentence of type str. This tells the module it will receive a sentence as input.
sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()
Declares an output field called sentiment whose type is a Literal limited to the values "positive", "negative", or "neutral". This instructs DSPy to parse and return one of those labels.
confidence: float = dspy.OutputField()
Declares another output field named confidence of type float. This tells DSPy to return a numeric confidence score alongside the label.

What this signature does overall:
Together, the fields form a function-like contract: given a sentence string, the module will return a sentiment label and a confidence float. DSPy uses this signature to (1) build the underlying prompt, (2) call the LLM, and (3) parse & cast the model output into the declared typed fields — returning a structured result instead of a raw string.

Running the Code

Now we'll use this function:

if __name__ == "__main__":
    set_model()
    dspy.configure()

    classify = dspy.Predict(Classify)
    result = classify(sentence="This book was super fun to read, though not the last chapter.")
    print("\nClassification Example:")
    print(result)

We first use the DSPy Predict module and pass in our Classify class to it and we get back a new classify function:

classify = dspy.Predict(Classify)

Now we can use the classify method by calling it and passing in a sentence parameter, just like we specified in our class.

result = classify(sentence="This book was super fun to read, though not the last chapter.")

The sentence we're trying to classify is "This book was super fun to read, though not the last chapter." This is positive for the first half and negative for the second half. So, unsurprisingly, when I run this code I get back a result like this:

Classification Example:
Prediction(
    sentiment='neutral',
    confidence=0.8
)

Gemini is 80% confident that this is a neutral sentence.

However, let's play around with this a bit and we'll see the dangers of not fully controlling your prompts. Let's intentionally try to screw things up a bit by redoing Classify like this:

class Classify(dspy.Signature): """Classify sentiment of a given sentence.""" sentence: str = dspy.InputField() sentiment: Literal["positive", "negative", "neutral", "boo!"] = dspy.OutputField() confidence: float = dspy.OutputField()

I added "boo!" as a possible sentiment, which makes no real sense. Now run the code again and we get:

Classification Example:
Prediction(
    sentiment='positive',
    confidence=0.75
)

By adding "boo!" as an option, we went from 80% confident this was a neutral statement to 75% confident it is positive. Why? I have no idea and I doubt the LLM does either.

How Prompts Are Built

You might, at this point, be wondering how this function is turned into a prompt that is sent to Gemini. The answer is that it uses the name of the class, the class properties, and even the docstring to come up with a prompt. To prove this, let's change the Classify method to instead be:

class Classify(dspy.Signature):
    """Return "boo!" as sentiment every time"""
    sentence: str = dspy.InputField()
    sentiment: Literal["positive", "negative", "neutral", "boo!"] = dspy.OutputField()
    confidence: float = dspy.OutputField()

Notice that all I did was change the docstring to tell it to return "boo!" every time. And we now get:

Classification Example:
Prediction(
    sentiment='boo!',
    confidence=1.0
)

Nice, eh? The docstring isn't just a comment any more, it's part of how you code the function!

Conclusion

Hopefully this short demo will help you understand how DSPy turns its classes/functions into prompts. There is clearly a lot of power here, but also some danger. The best reason to do this is if you plan to use DSPy's optimizers to let your software come up with the testably best prompts. Imagine how you might simply unplug one model, plug in another, then rerun the optimizer. You could easily move from one LLM to another using the same code base!

DSPy: Prompt your LLM Like It's Code

Tue, 02 Dec 2025 12:00:00 -0700

DSPy (pronounced "dee-s-pie") is an open-source Python framework that lets you exchange prompt building for code. Sounds impossible, right? What it really does is cleverly turn Python code into prompts behind the scenes. You get to specify what you want using code. For example, you might define a Classify function with specific inputs and expected outputs. Behind the scenes, DSPy converts that into a prompt, sends it to your Large Language Model (LLM), and returns a result formatted according to your specifications. The end result feels like you wrote code instead of building natural language prompts.

Why DSPy Instead of Prompt Building?

Why might you want to do this? Well, first of all, maybe you don’t. I often like to have full control over everything, so when I don’t know what’s being sent to the LLM, I’m like a nervous public speaker in front of a large auditorium—without being told what my topic is.

However, think about this differently. Using DSPy forces you to treat your interactions with the LLM as if they were functions with defined inputs and outputs. It formalizes these interactions into Python functions that are independent of the underlying LLM. That alone makes it valuable—but the real power comes when you use DSPy to optimize those interactions.

Prompt optimization in DSPy works by running your program on a set of developer-provided training examples and collecting traces of input/output behavior. It then systematically proposes and evaluates new prompt instructions (and optionally fine-tunes the model itself), as well as few-shot examples, to maximize your chosen metric. Yes, you heard that right! Instead of writing each prompt and relying on your gut (or ad hoc testing), DSPy will take training examples and try out different prompts until it finds the ones that perform best.

Even if you don’t use DSPy’s optimizers, there’s still value in the framework thanks to its pre-built modules. In this post, I’ll show you how to build a basic chain-of-thought model without having to write your own chain-of-thought prompts.

Getting Started with DSPy

First install DSPy:

pip install -U dspy

Now here is how to use the DSPy Chain of Thought module. "Chain of thought" is a prompting technique for an LLM that tells it to think through the problem step-by-step and reason when giving an answer. It was invented by Google in a famous 2022 paper called "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models". It allows an LLM to reason better. This example is a modified version of the the 'getting started' tutorial from the DSPy home page. I have modified it to use Google's Gemini as a model, which was basically just plug and play. I also added my standard get_secret function, as found here. It allows you to load a password out of a file of your choosing to avoid accidentally checking in an API key or password into your code base.

import dspy
from general_utils import get_secret

def set_model():
    # Load Gemini secret and configure LM
    gemini_secret: str = get_secret(r'D:\Documents\Secrets\gemini_secret.txt')
    lm = dspy.LM("gemini/gemini-2.5-flash", api_key=gemini_secret)
    dspy.configure(lm=lm)
    return lm

def chain_of_thought():
    math = dspy.ChainOfThought("question -> answer: float")
    result = math(question="Two dice are tossed. What is the probability that the sum equals two?")
    print(result)

if __name__ == "__main__":
    set_model()

    print("Chain of Thought Example:")
    chain_of_thought()

You can find my full code here.

Understanding the ChainOfThought Signature

Here we're using the ChainOfThought module:

math = dspy.ChainOfThought("question -> answer: float")

This signature string ("question -> answer: float") is how DSPy describes the inputs and outputs of a module in a compact, human-readable way.

The part before the arrow (question) names the input parameter the module expects.
The part after the arrow (answer: float) names the output and declares its type (float in this case).

Why this matters:

Prompt generation: DSPy uses that signature to automatically build the underlying prompt. Instead of you writing the prompt text, DSPy knows you expect a question and that the final answer should be a float, so it steers the model to produce a suitably formatted response.
Parsing & validation: After the LLM replies, DSPy parses and casts the model output into the declared type. In the example above, the returned answer is converted to a Python float.
Structured traces: For chain-of-thought modules in particular, DSPy usually returns both a reasoning trace (the step-by-step explanation the model produced) and the typed answer. That’s why your output looked like:

Chain of Thought Example:

Prediction(
        reasoning='When two dice are tossed, each die can show a number from 1 to 6.\nThe total number of possible outcomes is 6 (for the first die) * 6 (for the second die) = 36.\nTo find the sum that equals two, we need to list the combinations:\nThe only combination that results in a sum of two is (1, 1).\nThere is only 1 favorable outcome.\nThe probability is the number of favorable outcomes divided by the total number of possible outcomes.\nProbability = 1/36.\n\nTo express this as a float: 1 / 36 = 0.027777777777777776\nRounding to a reasonable number of decimal places, or just providing the direct float value.',
        answer=0.027777777777777776
)

Notice that the reasoning trace is how the LLM reasoned (via the chain-of-thought technique) to come up with an answer.

Conclusion

We have introduced DSPy and explained why you might want to treat your prompt building as code. We've installed it and used its out-of-the-box chain-of-thought module to show what you can do with minimal programming. Next time we'll look deeper at how DSPy works.

Google Gemini 3 Makes a Huge Leap on the ARC‑AGI Benchmark

Tue, 25 Nov 2025 09:00:00 -0700

Google’s Gemini 3 has posted a standout result on the ARC Prize Leaderboard – ARC‑AGI‑1, scoring about 87.5% in its Deep Think preview — a very strong showing on a benchmark focused on abstract reasoning. ARC‑AGI‑1 (Abstraction & Reasoning Corpus for AGI) is designed to test fluid intelligence: each task provides a few input/output examples, and models must infer underlying rules rather than rely on memorization. The benchmark emphasizes skill-acquisition efficiency, rewarding reasoning and generalization over brute-force performance. (ARC Prize Technical Report)

This 87.5% score suggests Gemini 3 Deep Think is effectively reasoning with abstract rules, not just pattern matching, and positions it as a model capable of structured, AGI‑style problem solving. Beyond ARC‑AGI‑1, Google has publicly confirmed that Gemini 3 Deep Think achieves 45.1% on ARC‑AGI‑2, a more challenging follow-up benchmark that also tests reasoning and code execution skills. (Google Blog, VentureBeat)

If these results hold, they represent a meaningful step forward: Gemini 3 is not just a larger LLM but a reasoning-first model capable of abstract problem solving. High performance on ARC‑AGI‑1 signals efficient learning and generalization — core aspects of intelligence that many benchmarks don’t test — and marks a clear signal that AI systems are beginning to handle tasks previously out of reach for conventional models.

AI Tutorial: Build a Free Gemini AI Chat Agent with n8n

Tue, 18 Nov 2025 12:00:00 -0700

n8n is a great tool for building useful AI Agents with minimal coding. And it even has a free version available if you want to self-host.

One of the first tutorials n8n offers is building an n8n chat agent. Let’s go through that tutorial but with a twist: we’ll connect to Google’s Gemini instead of OpenAI. This is consistent with Mindfire’s goal of finding the best low-cost AI resources for smaller clients that can’t afford a huge AI bill but want to add AI to their applications. Plus, my version of the tutorial will walk you through step-by-step visually to make it as easy as possible.

If necessary, install n8n as discussed in the previous post. That post got you running a free working version of n8n running locally. Once that is done, you’re ready to create your first chat agent.

Creating a New Workflow

After running n8n locally (for me that is http://localhost:5678/) and registering or signing in (as covered in the previous post) you’ll see the overview screen.

Click on “Create Workflow”:

Adding a Chat Trigger

You’ll be taken to the workflow building screen where you’ll click on “Add first step…”:

Next you should search for ‘chat trigger’ and select it:

From here you just want to go “Back to canvas”:

Adding an AI Agent

You’ll go back to the workflow screen where you will want to add another node:

Now search for “AI Agent”:

This will take you to the Edit AI Agent View. We need to add a chat model to the chat agent, so select the “+” under the ‘chat model’:

This will automatically take you back to the search screen but filtered on language models (notice the yellow box). You will search for the model of your choice. For this free demo, we’ll use Google Gemini so that it doesn’t cost anything:

The end result should look something like this the below. Notice that for me it defaulted to the right Gemini credentials because I’ve set them up before. It’s smart enough to reuse them. Also note that control to edit the credentials:

However, if you are doing this the first time, you’ll need to setup your credentials:

This will take you to a screen to setup credentials:

Host for Gemini is https://generativelanguage.googleapis.com and you’ll need an API Key which you can get from https://aistudio.google.com/. (See my tutorial for Google’s AI Studio here.)

There is a link to get an API key:

Now fill it into the credentials and it should work. I’ve had some problems with it not initially recognizing a valid Google model. Use models/gemini-2.5-flash (which should be the default). If it doesn’t like that and claims its invalid, I found it worked if I just went back to canvas and started over.

Connecting the Chat Trigger

It should automatically connect to the chat trigger by default, but I found I sometimes have to connect it manually like this:

It should now look something like this:

Adding Persistence

Large Language Models, by default, have no persistence. Each time you type to them they have no idea what you previously were talking about. To fix this problem, we’ll need to add ‘memory’ by clicking the “+” under ‘memory’ and then select ‘simple memory:

The default context window of 5 messages is fine for now, so go back to canvas:

You should have a completed workflow that looks like this:

One mistake I’ve made in the past is accidentally pinning the AI Agent so that it doesn’t execute when run. You can bang your head trying to figure out why your workflow isn’t working when really, it’s in test mode always returning default data. Be sure the pin icon is unselected here:

Let’s add a ‘system message’ which is just instructions for the chatbot to follow that allows you to lock in a specific style of chatbot:

In this case let’s have the chatbot act like a politician that no matter what you ask it finds a way to change the subject.

Save the final workflow

Chatting with Your AI Agent

Click the “Open Chat” button to test it out.

Try asking your chatbot something and see how it responds. For me, it started off with:

“Ah, the weather! A truly fascinating subject, wouldn't you agree? But you know, what's even more fascinating is the incredible progress we're making on the new community initiative. …”

And then it went on to change the subject to talk about its own political initiatives.

Conclusion

And that’s it! You’ve just built your first n8n workflow — for free. But this is only the beginning.

n8n is a powerful tool for automating almost anything. Imagine all those repetitive tasks you dread every day—gone. Need to schedule meetings automatically? Connect an AI agent to your calendar and email, and let it handle the back-and-forth for you. Overwhelmed by your inbox? Set up an AI-powered workflow to read messages, reply to urgent ones, and even text you when something truly needs your attention.

With n8n and AI, you can turn busywork into background work—so you can focus on what really matters.

Ready to see what else is possible?

AI Tutorial: Installing n8n Self Hosted Community Edition

Tue, 11 Nov 2025 12:00:00 -0700

I’m going to do some blog posts about using n8n to build an AI workflow. Consistent with our ‘open sourced’ / low-cost approach to Artificial Intelligence, I’m going to use the self-hosted community edition of n8n. This blog post will walk you through how to do the installation, in my case, for Windows. (Though it should work more or less the same for other operating systems.)

What is n8n?

n8n is a workflow-automation platform: it lets you connect apps, services, databases and APIs together, and automate sequences of tasks. It has a visual interface that lets you connect nodes together in a workflow graph:

This may sound somewhat similar to Deepset’s Haystack and there is some overlap of functionality. For example, Haystack also has a graphical interface available to create its pipelines and it can also chain together nodes that call services. However, n8n can be used for any kind of workflow because n8n offers extensive API integrations to services (like Slack or Google Sheets) whereas Haystack is more oriented specifically to integration with tools like PostgreSQL and pgvector or elastic search as well as various Large Language Models. In other words, n8n is more general purpose and Haystack is more oriented towards AI.

Consistent with Mindfire’s goal of finding low-cost AI solutions, I am going to go over how to install the self-hosted version of n8n using the free community edition. (Github repo found here.) Though other versions are available including a paid plan hosted by n8n. This edition uses a fair-code license, so it is free to use.

If you want to learn more, this quick start guide is a great place to start. The n8n website includes a number of text and video courses to bring you up to speed.

Installing n8n Locally

First, if you don’t already have it installed, you’ll need Node.js. See this blog post for details on how to install node.js.

Next, we need to install n8n itself. Go to this page for details.

I’ll walk you through my own install of n8n onto a Windows machine.

For Windows, open “Terminal” to get a Powershell command prompt.

For our purposes we want to do the global install, so at the command prompt type:

npm install n8n -g

You’ll see an install take place:

Notice at the end that you have a localhost url:

http://localhost:5678

Run that in a browser and you’ll get the n8n web interface running locally and you’ll get a registration screen:

Go ahead and register and create a password. There will be a few extra screens you’ll have to pass through the first time such as answering a survey:

Finally, you’ll get to the actual n8n screen:

You have now installed n8n and you’re ready to go with the community edition.

How to Install Node.js (for Windows)

Tue, 04 Nov 2025 12:00:00 -0700

I’m going to explore n8n in future blog posts, so, go over how to set up an AI workflow. To keep this consistent with our ‘open sourced’ approach to AI, we’re going to do a self-hosted version of n8n using the free community edition. To install n8n you need to have Node.js already installed so that you can use the npm command. So, for this blog post we’re going to briefly go over how to install Node.js. This one is particularly easy to do and likely you already have it installed for some other purpose. But for completeness, let’s quickly cover it:

Update: It's actually recommended that you use tools like nvs (Windows) and nvm (Mac & Linux)

Additionally, there are very few (if any) packages that should be installed globally. Installing those as local dependencies can make a huge difference in your ability to move between projects that have different node versions and requirements. You can do this with the command parameter --save-dev.

Example:

>nvs use 20.15
>node --version
v20.15.0
>npm install typescript --save-dev

First, go to the node.js home page:

https://nodejs.org/

You’ll see this page below. Click on the “Get Node.js” button:

That will take you to the download page (or you can just click the link below and go directly there):

https://nodejs.org/en/download

The download page offers you several different ways to download Node.js. Since we’re a Windows shop (and since Windows needs more AI love) I’m going to show you how to download the Windows installer, though feel free to change this to whatever operating system you prefer. First you need to select the operating system of your choice, which for me is Windows 64-bit:

Then you click the “Windows Installer.msi” button to do the download:

After the download, click the browser download icon and select what you just downloaded and run it:

This starts the installer:

Now just take the defaults in the installer and you’re ready for the next step for installing n8n.

Cox's Theorem: Is Probability Theory Universal?

Tue, 21 Oct 2025 12:00:00 -0600

Most of life involves reasoning using incomplete information. Or put another way, it's about reasoning under uncertainty. Will it rain tomorrow? Should you carry an umbrella? Classical (propositional) logic handles things that are exactly true or false — but it doesn’t tell you how to reason when you’re unsure.

In our last several posts, particularly this one, we talked about how probability theory is a generalization of boolean and propositional logic. That is to say, probability theory extends regular deductive logic and includes it; at least for boolean and propositional logic. (There is still an open question on if this is also true for First Order logic.) So probability theory gives us a way to reason with incomplete or uncertain information.

But is probability theory just one possible way to reason under uncertainty? Or is it the only way to do so?

Cox’s theorem (R. T. Cox, 1946) attempts to answer that question. Put simply: if you want a system for plausible reasoning that is sensible and consistent, you are inevitably led to probability theory. Edwin T. Jaynes championed and popularized this view; see his book (Probability Theory: The Logic of Science).

If you want to dive into the originals, Cox’s paper and notes are available online:

What we Demand of a Reasoner (According to Jayne)

Jaynes summarized Cox’s intuition in three crisp requirements. Jayne calls these 'desiderata' because they aren't axioms assumed to be true, but rather desires or goals we wish to constrain ourselves to. Despite the simplicity and even apparent 'obviousness' of these 'desiderata', these are the only things we ask of a system that assigns degrees of plausibility to statements:

(I) Representation — degrees of plausibility are real numbers.
You should be able to say, “this is more plausible than that” and encode that judgment with a single number:

Everything gets a number. Any statement A has a plausibility pl(A) (think: a slider).
Negation is linked. Knowing pl(A) should tell you pl(not A) (they’re not independent).

The first requirement means you can always compare two (mutually exclusive) statements and say one is more plausible than the other, or they're equally plausible. On the surface this seems rather reasonable and almost inevitable. How else could plausibility work but via some way to ultimately decide between mutually exclusive options? And that implies there must be some way to weigh the options compared to their competitors. This isn't possible if you can't translate everything down to a single continuous value.

The second sub point seems intuitively correct because knowing how plausible a statement is, you should automatically know how plausible its negation (the opposite statement) is. Suppose you think “it will rain” is somewhat likely: pl(rain) = 0.8 (80% plausible). Then, under Cox-like rules, the plausibility of “not rain” is determined: pl(not rain) = 0.2. If you also learn “it’s cloudy” and that makes “rain” more plausible, the rules tell you precisely how to combine the old plausibilities and the new evidence — and that combination must behave like Bayesian updating (i.e., use conditional probabilities).

Despite the 'obviousness' of this desiderata, this is probably the most challenged requirement.

(II) Qualitative correspondence with common sense.
When information is crisp, the system should reduce to ordinary logic; small changes in evidence cause small changes in plausibility:

Respect logic: logically equivalent statements get the same plausibility.
Continuity: tiny changes in input → tiny changes in output.
The AND rule (decomposability): the plausibility of “A and B” should depend only on how plausible B is, and how plausible A would be if B were true.

The continuity requirement seems sensible because you wouldn't expect a tiny piece of new information to cause a massive, sudden jump in your confidence about something.

(III) Consistency.
Different, valid routes to the same conclusion must give the same answer; the rules must scale and compose:

Universality: the same rules must work in any domain.
Non-contradiction: if a conclusion can be derived more than one way, all derivations agree.
Scalability: rules that make sense for one case should still make sense for many repeated or combined cases.

Your system of plausible reasoning should be consistent like formal logic (for our purposes that means propositional logic). For example, if two statements are logically equivalent (they mean the same thing), they should always have the same plausibility. Likewise, if something is always true by definition (a "tautology"), its plausibility should be at the maximum possible value.

And the system should be universal – it shouldn't just apply to a specific type of problem or domain. You should be able to reason about anything, from unrelated events to complex scenarios, and the underlying rules should still hold. This is crucial for making the logic broadly applicable, just like the fundamental rules of mathematics or logic apply everywhere.

One Probability Logic to Rule Them All?

Cox showed — and later expositions (Jaynes, others) made rigorous — that those seemingly mild, intuitive demands uniquely determine the algebra of plausible reasoning. Up to a simple re-scaling, the plausibility numbers must obey the product rule and the sum/negation rule. After mapping to the usual 0–1 scale, those rules are exactly the familiar axioms of probability:

P(A and B | C) = P(A | B,C) × P(B | C) (product rule)
P(A | C) + P(not A | C) = 1 (negation/sum rule)

It is possible to come up with alternatives that might on the surface look a lot different. But there will always be a function that maps the results back to simple probability theory. The only alternative to this is a system that doesn't match those seemingly innocuous desiderata. (Which seems rather undesirable.)

In short: If you accept Jaynes desiderata, probability theory is the only consistent extension of Boolean logic to uncertain situations. Cox and Jayne argue that probability theory is not merely a tool for frequencies or gambling — it’s the uniquely rational way to handle degrees of belief in uncertain situations.

So Why Does This Matter?

Cox's theorem is used to argue for several important points:

Foundational justification for Bayesian reasoning. Cox gives a principled reason to treat probabilities as degrees of belief (not only long-run frequencies).
Objectivity and consistency. If you accept the desiderata, then any other scheme will give contradictory or “nonsensical” answers in some cases.
Applies to single events. You don’t need repeatable trials — probability applies to unique hypotheses (e.g., “life exists on other planets”) because it codifies consistent belief, not frequency.
Practical payoff. The rules Cox forces on us are exactly the ones used in Bayesian inference, decision theory, and much of modern machine learning.

Conclusion and A Quick Caution

There are ongoing technical discussions (and a few edge-case counterexamples) in the literature about which exact axioms are needed. There are also a few (not so well known) alternative theories of plausibility that try out different desiderata -- such as not requiring plausibilities to be put in terms of a single real value.

Still — this seems like a big breakthrough. It takes the idea we previously proved — that probability theory is an extension to propositional logic — and formalizes it and proves that under seemingly reasonable assumptions probability theory is the sole and only way to represent logical plausibilities.

It is not an accident that Bayesian probabilities have come to dominate Machine Learning in recent years.

From Certainty to Belief: How Probability Extends Logic - Part 3

Tue, 14 Oct 2025 09:00:00 -0600

Now we pull all those threads together and formally show that probability theory is an extension of propositional logic.

As a reminder, in earlier posts we built a sequence of ideas:

Aristotle: The First AGI Researcher - We covered the long road from Aristotle to today to try to simulate human thinking that led to the creation of deductive logics. In that post we also included a primer on propositional logic, the basis for computation.
The Atomization of Thought — We showed any boolean function can be built out of nothing more than AND and NOT operators by using 'minterms'. That is to say, AND + NOT are universal operators that can represent any other boolean operations or function of any complexity.
One Logical Operator to Rule Them All — We showed that a single operator, NAND (or the pair NOT+AND together), is functionally complete: every propositional function can be built from them. It is a single universal boolean operator.
From Certainty to Belief: How Probability Extends Logic - Part 1 and Part 2 — We introduced variables, marginals, conditional probability and Bayes’ Rule, and showed how probabilistic reasoning generalizes syllogistic logic as created by Aristotle.

Now we pull all those threads together and formally show that probability theory is an extension to propositional logic.

Because NOT and AND (or NAND) are enough to express any propositional formula, it is sufficient to show how to express NOT and AND in probability theory. Once we can do that, probability theory can represent every propositional logical function — with logical entailment appearing as probability = 1.

How to encode NOT and AND with probability

Encoding NOT and AND boolean operators into probability theory is remarkably easy. We treat logical statements as random variables taking values tr (true) or fa (false). Then we use probability constraints to encode logical facts.

NOT
Logical ¬A (not A) is encoded by the probabilistic constraint: p(A = tr) = 0.

But recall that for two mutually exclusive values for a random variable they must marginalise (sum) to 1. So if p(A = tr) = 0 then p(A = fa) = 1 - p(A = tr). Since p(A = tr) = 0, this means:

p(A = fa) = 1 - p(A = tr) = 1 - 0 = 1.

So p(A = fa) = 1. This forces A to be false in every model/distribution that satisfies the KB. This allows us to represent the NOT operator using the notation of probability theory.

AND
Logical A ∧ B (A and B) can be expressed in probability in a couple of very simple, equivalent ways — pick whichever feels more natural.

Joint form (one statement): say the probability that both A and B are true is 1:
p(A = tr, B = tr) = 1
This literally means: in every scenario allowed by our assumptions, A and B are both true.
Separate form (two statements): say A is true with certainty and B is true with certainty:
p(A = tr) = 1 and p(B = tr) = 1
Together these mean the same thing as the joint form — the only possible worlds left are those where A and B are both true.

Either way, you have encoded the logical AND using probabilities.

NAND (single-operator route)
NAND(A,B) is just NOT (A ∧ B). To say “A NAND B is true” you can rule out the assignment where both A and B are true:

p(A = tr, B = tr) = 0

That says: the situation where both A and B are true has zero probability — it cannot happen. Since NAND is universal, you can use this single kind of constraint to build any Boolean function if you want to work from one operator only.

Tiny worked example — A ∧ ¬B

We want to encode the formula “A and not B”.

Two equivalent, simple encodings:

Joint constraint: require the single assignment where A is true and B is false to be the only possibility:
p(A = tr, B = fa) = 1
Read plainly: the only allowed scenario is A true and B false — so the formula is true in every allowed scenario.
Atom constraints: say A is certain and B is impossible:
p(A = tr) = 1 and p(B = tr) = 0
That also leaves only worlds where A is true and B is false, so again the formula is true in every allowed scenario.

Either approach works.

Logical connectives — Boolean version and probabilistic version

NOT (¬)
Boolean: If P is true, then ¬P is false; if P is false, then ¬P is true.
Probability: encode ¬P by forcing P to be false:
p(P = tr) = 0 (equivalently p(P = fa) = 1).
AND (∧)
Boolean: P ∧ Q is true only if both P and Q are true.
Probability (joint form): require both to be true with certainty:
p(P = tr, Q = tr) = 1.
Probability (separate form): require each true with certainty:
p(P = tr) = 1 and p(Q = tr) = 1 (together these imply the joint form).
OR (∨)
Boolean: P ∨ Q is true if at least one of P or Q is true.
Probability (direct): require that at least one is true with certainty:
p(P = tr or Q = tr) = 1 (read literally: the probability that P or Q is true equals 1).
Probability (algebraic): using inclusion–exclusion:
p(P = tr) + p(Q = tr) - p(P = tr, Q = tr) = 1.
IMPLIES (→)
Boolean: P → Q is false exactly when P is true and Q is false; otherwise it’s true.
Probability (for implication-as-certainty): either of these equivalent encodings works:
- Forbid the counterexample: p(P = tr, Q = fa) = 0.
- Or express as a conditional: p(Q = tr | P = tr) = 1.
  (Both say: whenever P is true, Q must also be true. However, when using conditional form p(Q = tr | P = tr) = 1, this implicitly assumes p(P = tr) > 0; if p(P = tr) = 0 prefer the joint form p(P = tr, Q = fa) = 0.)
EQUIVALENT (↔)
Boolean: P ↔ Q is true only when P and Q share the same truth value (both true or both false).
Probability (joint form): require that only the matching pairs are true:
p(P = tr, Q = tr) + p(P = fa, Q = fa) = 1.
Probability (conditional form): equivalently, require both directions as certain:
p(P = tr | Q = tr) = 1 and p(P = tr | Q = fa) = 0.

Note that: - Saying p(...)=1 is the probabilistic way to say “this is certainly true” (the logical 1).
- Saying p(...)=0 is the probabilistic way to say “this cannot happen” (the logical 0).
- You can mix joint probabilities (e.g., p(P,Q)) and conditionals (e.g., p(Q | P)) — they are equivalent ways to express the same logical constraint.
- Once you can express NOT and AND this way, you can build OR, IMPLIES, EQUIV, and therefore any propositional formula, by combining the probabilistic encodings above.

Conclusion

As this post shows, probability theory can represent any boolean or propositional logical statement. Thus probability theory extends boolean and propositional logic. That is to say, it generalizes boolean and propositional logic — the 0/1 case of probability theory recovers ordinary logic.

But what about First Order logic? Also, is there a way to formally prove this?

From Certainty to Belief: How Probability Extends Logic - Part 2

Tue, 07 Oct 2025 09:00:00 -0600

In our last post, we built an intuition for why probability theory might be thought of as an extension of deductive logic. In an earlier post we went over the basics of probability theory. (And in an even earlier post, we went over the basics or propositional logic.)

Let's now see if we can formalize our intuition and show that it is possible to do deductive logic using only probability theory. To do this, I'm going to use two fun examples taken from David Barber’s textbook Bayesian Reasoning and Machine Learning (we follow his Example 1.5 and 1.6 on p. 11-12). We'll work them through his examples step by step, explicitly showing which rules or probability theory we use at every line. The goal is to make the connection between logic and probability completely transparent.

Quick reminder of the rules of probability theory

Rules all taken from this post.

Variables & states
A variable (e.g. A, F, T) can take tr (true) or fa (false).
Joint symmetry
p(x, y) = p(y, x). The joint probability is symmetric.
Conditional probability (definition)
p(x | y) = p(x, y) / p(y) — equivalently p(x, y) = p(x | y) × p(y).
Marginalisation (summing out)
p(x) = sum_y p(x, y) — in two states: p(x) = p(x, y=tr) + p(x, y=fa).
Bayes’ Rule (derived from the above)
p(x | y) = [p(y | x) × p(x)] / p(y).
Conditional independence (when stated)
p(z | x, y) = p(z | y) means once y is known, x gives no extra info about z.

Example 1 — Aristotle’s syllogism (Barber Example 1.5)

Informal statement:

“All apples are fruit."
"All fruit grow on trees."
"Therefore, all apples grow on trees.”

Variables:

A = "is an apple" (A = tr means it is an apple)
F = "is a fruit"
T = "grows on trees"

Given (probabilistic form of the premises):

p(F = tr | A = tr) = 1 (All apples are fruit.)
p(T = tr | F = tr) = 1 (All fruits grow on trees.)
Conditional independence assumption: p(T | A, F) = p(T | F).

Note that #3 means that if you know fruit and apples grow on trees that since an apple is a fruit this is the same as saying just fruit grows on trees. We're simply showing that you learn nothing more given A if you already have F.

Goal: show p(T = tr | A = tr) = 1.

That is to say, show that if you know something is an apple, you know it grew on a tree given your premises. (Exactly like deductive logic.)

Step-by-step derivation (every step labeled with the rule used):

Start with the target: p(T = tr | A = tr).
(Goal)
Use marginalisation over F (rule 4):
p(T = tr | A = tr) = Σ_s∈{tr,fa} p(T = tr, F = s | A = tr).
(We sum over the unobserved variable F to account for both states s = tr and s = fa.)
Use the definition of joint/conditional (rule 3):
p(T = tr, F = s | A = tr) = p(T = tr | F = s, A = tr) × p(F = s | A = tr).

Therefore:
p(T = tr | A = tr) = Σ_s p(T = tr | F = s, A = tr) × p(F = s | A = tr).

Apply the conditional independence assumption p(T | A, F) = p(T | F) (given):
Replace p(T = tr | F = s, A = tr) with p(T = tr | F = s).

Now:
p(T = tr | A = tr) = Σ_s p(T = tr | F = s) × p(F = s | A = tr).

Write out the explicit two-term sum (s = fa and s = tr):
p(T = tr | A = tr) = p(T = tr | F = fa) × p(F = fa | A = tr) + p(T = tr | F = tr) × p(F = tr | A = tr).
Use premise 1: p(F = tr | A = tr) = 1. By normalisation, p(F = fa | A = tr) = 0.

This is because probability of all possible events must sum to 1 under probability theory. So if p(F = tr | A = tr) = 1 then its inverse p(F = fa | A = tr) = 0. Or put in plain English, if it is certain that something being an apple is always a fruit too, then it must be equally certain that given something is an apple it must be false that it is not a fruit. (Double negative there.)

Substitute:
p(T = tr | A = tr) = p(T = tr | F = fa) × 0 + p(T = tr | F = tr) × 1.

Use premise 2: p(T = tr | F = tr) = 1.
Since the first term is multiplied by 0, p(T = tr | F = fa) drops out, leaving:
p(T = tr | A = tr) = 0 + 1 × 1 = 1.

Note that we do not need to know what the probability p(T = tr | F = fa) (that is, the probability if you know something isn't a fruit that it grows on a tree) to resolve this problem thanks to it dropping out.

Conclusion (Example 1): p(T = tr | A = tr) = 1.
In words: given the premises and conditional independence, probabilistic rules yield the conclusion: "all apples grow on trees" just like the classical logical syllogism.

Barber points out that this reasoning is a form of resolution, a kind of transitivity: from A ⇒ F and F ⇒ T, we can infer A ⇒ T.

Or put simply, it is possible to use the probability calculus to do regular logical resolution and work out a syllogism.

Example 2 — The contrapositive (Barber Example 1.6)

This example is important because Karl Popper noted that the contrapositive (aka Modus Tollens) is the basis for the logic of scientific reasoning. (That we 'falsify' our theories rather than verify them.) So showing that we can do the same thing with probability theory will show that probability theory is also the logic of science. (Or at least can be.)

Informal statement:

"If A then B" implies "If not B then not A."

Variables: - A and B are boolean variables (tr/fa).

Given (probabilistic premise): - p(B = tr | A = tr) = 1. (If A is true then B is certainly true.)

Goal: show p(A = fa | B = fa) = 1. (That is, show that given B to be false that you know for sure that A is also false.)

Step-by-step derivation:

Start from normalization for A conditioned on B = fa:
p(A = fa | B = fa) = 1 - p(A = tr | B = fa).
Because p(A=fa | B) + p(A=tr | B) = 1. (i.e. the total of all possibilities must sum to 1)
We will show p(A = tr | B = fa) = 0. If that holds, the left side equals 1.
From the premise p(B = tr | A = tr) = 1, take the complement to get:
p(B = fa | A = tr) = 1 - p(B = tr | A = tr) = 1 - 1 = 0.
(If A is true, B cannot be false.)
Now use Bayes’ Rule to express p(A = tr | B = fa) in terms of p(B = fa | A = tr):
p(A = tr | B = fa) = [ p(B = fa | A = tr) × p(A = tr) ] / p(B = fa).
- The numerator contains p(B = fa | A = tr), which we just showed equals 0.
- Therefore the numerator is 0 × p(A = tr) = 0.

So the fraction equals 0 / p(B = fa) = 0 (provided p(B = fa) > 0). The denominator can be ignored. Note that if p(B = fa) = 0, then p(B = tr) = 1 and the contrapositive is vacuously satisfied anyhow because given p(B = tr) it must be the case that p(B = tr | A = tr) regardless.

Thus p(A = tr | B = fa) = 0. Plugging this into step 1:

p(A = fa | B = fa) = 1 - 0 = 1.

Conclusion (Example 2): p(A = fa | B = fa) = 1.

Or in other words, if B is false than A must also be false. This matches the logical contrapositive (modus tollens): from "If A then B" we infer "If not B then not A."

Why these worked and what they teach us

Every step used only the standard rules of probability theory: conditional probability, marginalisation (summing), joint expansion, Bayes’ rule and (where stated) conditional independence. I called out which rule was used at each line so you can trace the logic directly back to the primer.
When premises are certainties (probability = 1), probabilistic reasoning reproduces classical logic. In the examples above the premises used conditional probabilities equal to 1; that forces certain conditional and marginal terms to be 0 or 1 and collapses the algebra to the logical conclusions.
Probability can reproduce Aristotle's Syllogisms . We specifically choose to reproduce Aristotle's Syllogisms so that we could prove that probability theory was a generalization of syllogistic logic.
Probability is strictly more general than propositional or Boolean logic. If any premise had been less than 1 (for example p(B = tr | A = tr) = 0.9), the same algebra would produce posterior probabilities strictly between 0 and 1 instead of 0 or 1. That is the strength of the probabilistic approach: it handles degrees of belief as naturally as it handles certainties.

Conclusion

This demonstration goes beyond our previous intuition that probability theory extends classical deductive logic -- at least for syllogistic logic so far. But does this represent a formal proof? Can probability theory represent all of propositional logic? And what about First Order logic?

We will consider these questions in future blog posts.

From Certainty to Belief: How Probability Extends Logic - Part 1

Tue, 30 Sep 2025 09:00:00 -0600

In our previous post, we gave an introduction to probability theory. In an even earlier post, we went over the basics of propositional logic — starting with the logic of Aristotle, moving on to Boolean logic, and finally showing the link to propositional logic.

But is there any relationship between probability theory and deductive logic? In fact, there is! Boolean logic and propositional logic can be thought of as a special case of probability theory.

This may seem quite strange at first. When we think about reasoning, our minds often jump to logic: if certain statements are true, then other statements must be true. This is the bedrock of deductive logic — a system built on clear-cut, binary truths: something is either true or false.

What could logic possibly have to do with something as squishy and uncertain as probability theory? Think back to the examples we used in the previous post on probability theory. If you roll a die, you don't know for certain the outcome. But you do know that rolling two fair dice and scoring a total of 12 has only a 1 in 36 chance of happening. So if you needed to make a rational decision based on those odds, you could calculate the rationally correct choice — even though there’s no certainty like there is in deductive logic.

So just intuitively, there does seem to be something at least similar between deductive logic and probability theory. Both can be used to make rational decisions.

Given this intuition, could it be possible that there is some sort of formal relationship between deductive logic and probability theory? Or is this intuition misguided?

Deductive Logic: The World of Absolutes

To explore this further, let's start with traditional deductive logic (either propositional or Boolean logic works here), where statements are assigned one of two values: True or False. If "A implies B" is true, and "A is true" is true, then "B is true" must logically follow. There’s no room for "A is probably true" or "B is sometimes true." It’s a system of absolute certainties.

But our real world is rarely so black and white. We encounter situations where information is incomplete, observations are noisy, and conclusions are tentative. This is where probability theory offers a richer language that allows more flexible reasoning.

Probability Theory: Quantifying Uncertainty

Probability theory provides a framework for managing uncertainty. When you roll a fair die, you don't know what outcome you'll get — but you do have a rational explanation for why a 12 on two fair dice will only happen 1 in 36 throws. So you can assign a confidence to the outcome even though it’s uncertain.

In fact, this can also be true for a non-random event. In our previous post, we considered the “probability” of an asteroid hitting the Earth and even put a number on that event despite it being non-random. Re-imagining probability theory as a sort of plausibility calculus (more on this in future posts) allows us to use the same mathematical rigor for both uncertainty due to randomness in nature and uncertainty due to our ignorance.

Imagine an extension to propositional logic where, instead of only restricting yourself to the two values of 1 = True and 0 = False, you allow any continuous value between 0 and 1. If the value is 0.5, you have no reason to prefer assuming the event in question is true or false (say, you don’t know if a giant asteroid will hit the Earth this year). If the value is 1, you currently accept the event as true (you’ve observed an asteroid heading toward Earth and calculated it’s on a collision course). If the value is 0, you currently accept the event as false (you’ve scanned the sky and there are no asteroids on a collision course this year).

Now say you have no evidence to work with other than the fact that asteroids hit the Earth once every 500,000 years (as discussed in the previous post ). Then your best estimate is that there is a 0.0002% chance that a giant asteroid will hit the Earth this year.

Notice how allowing continuous values from 0 to 1 lets us mathematically express these intuitive ideas about the “probability” of a giant asteroid smashing into the Earth given some set of observations. This is true even though giant asteroids are not random events.

What, then, are we expressing if not the probability of a random event? There is considerable debate on this point, but the most obvious answer is that we’re expressing the plausibility of an event or statement being true given what we currently know — and what we currently don’t know. This is sometimes called, by Bayesians, degrees of belief (a term I don’t object to, even though I think Bayesians only have part of the truth about probability theory).

When viewed this way, it seems intuitive that probability theory is an extension of deductive logic. Deductive logic is just a special case of probability theory that allows only True (1) and False (0), whereas probability theory allows any value between 0 and 1.

Probability Theory as an Extension of Deductive Logic

Can we show formally that our intuition is correct — that deductive logic is a special case of probability theory, and that probability theory extends deductive logic?

In my next post, I’ll go over two excellent examples from David Barber’s Bayesian Reasoning and Machine Learning that demonstrate exactly that.

Dice Rolls, Coin Flips, and Death By Asteroid: A Probability Refresher

Fri, 26 Sep 2025 12:00:00 -0600

Introduction: Randomness, Belief, and Asteroids

Imagine flipping a coin or rolling a die—those feel truly random. Every flip and roll is fresh, unpredictable, and fair. When we say there is a 50% chance of getting heads or “1‑in‑6” odds of rolling a six, probability captures the genuine randomness of the experiment.

But what about something like Earth getting hit by a giant asteroid? That event doesn’t feel random in the same way—it’s a unique, one‑off cosmic occurrence based on deterministic laws. There is either an asteroid heading to earth that is going to hit us this year or there isn't. So where does probability come into play?

When we assign a probability to it, we’re really modeling our uncertainty—our lack of exact knowledge—not a repeatable random process like a coin flip.

For example, consider asteroids around 1 km in diameter—the kinds that could trigger widespread devastation or even global catastrophe. According to the Catalina Sky Survey, such an object strikes Earth on average once every 500,000 years (Catalina Sky Survey FAQ). That translates to an annual probability of approximately:

1 / 500,000 = 0.000002 (or 0.0002%)

In this case, probability is less about dice‑rolling-style pure randomness and more about representing our ignorance and expectations based on long‑term historical frequencies.

In the sections that follow, we’ll revisit classic probability concepts using coin flips and dice rolls—but we’ll also carry forward that perspective: probability is sometimes about real randomness, and sometimes about modeling how much we don’t know.

Variables and States

Let’s start with a simple example: flipping a fair coin. A coin can be in one of two states, either heads or tails.

We can represent this experiment using a random variable. Let’s call it c, for coin. The set of values that c can take is called its domain, and we write this as:

dom(c) = {heads, tails}

This means that the variable c can be in one of two states: heads or tails.

If the coin is fair, we assign equal probabilities to each outcome:

p(c = heads) = 0.5
p(c = tails) = 0.5

So when we say something like p(heads) = 0.5, we’re using shorthand for p(c = heads)— the probability that the coin lands on heads.

The important thing here is that we’re treating the coin as a variable (c), and the results it can take on (heads or tails) as states of that variable.

An event is an expression about a random variable such as two heads in a row or two heads out of three flips.

Summing Over Variables

Another important rule in probability is that the total probability over all possible states of a variable must add up to 1. This is called the normalisation condition. When you flip a coin you know that it must either land on heads or tails, so the total probability of both combined must add up to 1.0, meaning there is a 100% chance (certainty) it will be one of the two sides.

For our coin variable c, this means:

p(c = heads) + p(c = tails) = 1

Or, using more compact notation:

∑ p(c) = 1

This just says that when we consider all possible outcomes of the coin flip, one of them must happen. Either it lands heads, or it lands tails—there are no other options.

This idea of summing over all the states of a variable will show up again later when we start working with multiple variables.

In summary:

A variable represents an uncertain quantity—like the result of a coin flip.
The states of that variable are the specific outcomes it can take—heads or tails.
The domain is the full set of those possible states.
And the sum of the probabilities of all states of the variable must equal 1.

Interacting Variables: ANDs and ORs

Sometimes we’re interested in how two variables—or two events—relate to each other. One of the core rules in probability helps us figure out the chance of either event happening:

p(x or y) = p(x) + p(y) - p(x and y)

This accounts for the fact that if both events can happen together, we shouldn’t double-count that overlap.

Let’s look at an example with a 6-sided die.

Let Event A be: the die shows an even number → {2, 4, 6}
Let Event B be: the die shows a number greater than 3 → {4, 5, 6}

These events overlap at 4 and 6.

Now let’s calculate the probabilities:

p(A) = 3/6 = 0.5 (since 2, 4, and 6 are even)
p(B) = 3/6 = 0.5 (since 4, 5, and 6 are greater than 3)
p(A and B) = 2/6 ≈ 0.333 (since 4 and 6 satisfy both)

Using the formula:

p(A or B) = p(A) + p(B) - p(A and B)
p(A or B) = 0.5 + 0.5 - 0.333 = 0.667

So there’s about a 66.7% chance that the die roll is either even, or greater than 3—or both.

This rule becomes especially useful when we work with more complex combinations of events later on.

Marginal Probability

Now that we've introduced how to handle two variables at once, it might be helpful to see how to take multiple variables and reduce back to one by summing the other variables. This is called calculating the marginal probability.

Let’s say we flip two coins. We’ll call them:

c1: the result of the first flip
c2: the result of the second flip

Each coin has two possible outcomes: heads or tails.

The full joint distribution (probabilities of combined events) includes all possible combinations:

p(c1 = heads, c2 = heads) = 0.25
p(c1 = heads, c2 = tails) = 0.25
p(c1 = tails, c2 = heads) = 0.25
p(c1 = tails, c2 = tails) = 0.25

These probabilities all add up to 1, as expected.

Now let’s say we only care about the result of the first coin. We can compute its marginal probability by summing over all outcomes of the second coin:

p(c1 = heads) = p(c1 = heads, c2 = heads) + p(c1 = heads, c2 = tails)
p(c1 = heads) = 0.25 + 0.25 = 0.5

p(c1 = tails) = p(c1 = tails, c2 = heads) + p(c1 = tails, c2 = tails)
p(c1 = tails) = 0.25 + 0.25 = 0.5

This process is called marginalisation. We’re taking a joint distribution of multiple variables and extracting just the part we care about—by summing over the rest.

Conditional Probability

Sometimes we don’t just want to know the probability of an event—we want to know the probability given that something else has already happened.

This brings us to conditional probability.

We write this as:

p(x | y) = p(x and y) / p(y)
or more formally:
p(x | y) = p(x, y) / p(y)

This definition tells us how to update our probability of x when we know that y is true. The numerator is the joint probability that both x and y happen, and the denominator scales it relative to how likely y is overall.

Let’s look at a simple dice example:

Event A: the die shows a 6
Event B: the die shows an even number (2, 4, or 6)

We want to know:

p(A | B) — the probability that the die shows a 6 given that it shows an even number.

Using the formula:

p(A | B) = p(A and B) / p(B)
p(A | B) = p(6) / p(even)
p(A | B) = (1/6) / (3/6) = 1/3

So if we already know that the die landed on an even number, there’s a 1 in 3 chance it was a 6.

This is the essence of conditional probability: we’re updating what we believe about one event in light of new information.

Bayes’ Rule: Deriving It Step by Step

Let’s start with something simple but powerful:

🔁 Joint probabilities are symmetric

The joint probability of two events x and y is the same regardless of their order:

p(x, y) = p(y, x)

This just says: the probability of both x and y happening is the same as the probability of both y and x happening. They describe the same outcome.

📘 Now recall the definition of conditional probability

By definition:

p(x | y) = p(x, y) / p(y)
This gives us the probability of x given that y has occurred.

Likewise:

p(y | x) = p(y, x) / p(x)
This gives the probability of y given that x has occurred.

Let's calculate p(x, y) now by multiplying both sides by p(y):

p(x,y) = p(x|y) p(y)

Note also that we could have instead done this as:

p(y, x) = p(y|x) p(x)

🔁 Substituting using the symmetry of joint probabilities

We know from earlier that:

p(x, y) = p(y, x)

So we can replace p(x, y) in the first equation with p(y | x) × p(x) (from the second equation above):

p(x | y) = p(x, y) / p(y)
p(x | y) = [p(y | x) × p(x)] / p(y)

✅ Bayes’ Rule

We’ve now arrived at the full expression:

p(x | y) = (p(y | x) × p(x)) / p(y)

This is Bayes’ Rule. It allows us to reverse the direction of a conditional probability—from p(y | x) to p(x | y)— by incorporating:

p(x): our prior belief about x (i.e. the probability of x before we have any evidence)
p(y): the overall likelihood of observing y
p(y | x): how likely y is if x is true

Symptoms and Diseases: Using Bayes’ Rule to Flip the Question

Imagine you go to the doctor because you have a particular symptom—say, a persistent cough. Naturally, you want to know: What is the probability that you have a certain disease given this symptom?

This sounds straightforward, but here’s the catch: it’s often hard to directly measure how likely someone with that symptom actually has the disease. Symptoms can be caused by many things, and collecting data on all people with that symptom and their diagnoses is complicated.

However, it’s usually easier to measure two other things:

How often people who have the disease exhibit that symptom. For example, suppose that 80% of people with disease X have a cough. This is the probability of the symptom given the disease:

p(cough | disease) = 0.8

How common the disease is in the general population. For example, maybe disease X affects 1 in 1,000 people:

p(disease) = 0.001

With those two pieces of information, and using Bayes’ Rule, we can calculate what we really want:

What is the probability of having the disease given the symptom?

p(disease | cough) = [p(cough | disease) × p(disease)] / p(cough)

But what about p(cough), the overall probability of coughing? We can estimate that by considering:

The chance of coughing if you have the disease: p(cough | disease) = 0.8
The chance of coughing if you do not have the disease: say this is p(cough | no disease) = 0.1 (10% of people cough for other reasons)
The overall chance of having the disease: p(disease) = 0.001
The chance of not having the disease: p(no disease) = 1 - 0.001 = 0.999

To compute p(cough) — the overall probability that someone has a cough — we can break it down based on whether or not they have the disease. This follows the same logic we saw earlier with marginal probability, where we summed over the possible states of a variable.

In this case, we’re summing over two mutually exclusive groups: those with the disease and those without it. So we write:

p(cough) = p(cough, disease) + p(cough, no disease)

Using the definition of conditional probability

(p(x, y) = p(x | y) × p(y)),

we can rewrite this as:

p(cough) = p(cough | disease) × p(disease) + p(cough | no disease) × p(no disease)

This is also known as the law of total probability.

Substituting the values:

p(cough) = (0.8 × 0.001) + (0.1 × 0.999) = 0.0008 + 0.0999 = 0.1007

So even though the symptom is very common (10% of people get it regardless), the contribution from the rare disease (0.08%) is small in comparison.

Finally, apply Bayes’ Rule:

p(disease | cough) = (0.8 × 0.001) ÷ 0.1007 = 0.0008 ÷ 0.1007 ≈ 0.00795

This means:

Given that you have a cough, the chance that you have the disease is about 0.8% — or roughly 8 in 1,000.

Even though the disease is quite rare (0.1%), and the symptom common (coughing can be caused by many things), Bayes’ Rule helps us update the probability of disease in light of the symptom.

Conclusions

So what does this have to do with asteroids? I'm glad you asked.

Throughout this post, we've looked at how probability helps us reason under uncertainty:

We defined variables and their possible states, like coin flips and dice rolls.
We explored how probabilities add up across all possible outcomes of a variable.
We looked at joint probabilities and how multiple variables interact.
We introduced marginal probability by summing over hidden or unobserved variables.
And we learned how Bayes’ Rule lets us flip the direction of a probability—from something easier to measure (like symptoms given disease) to something we really want to know (like disease given symptoms).

But all of this isn’t just about games of chance.

An asteroid hitting Earth isn’t like flipping a coin—it’s governed by physics. But we still use probability because we don’t know everything: we might not know the exact location of all near-Earth objects, or the precision of their orbits decades from now.

In this case, probability reflects uncertainty in our knowledge, not randomness in the universe. And yet, the same mathematical tools apply:

We can define variables like “asteroid hits Earth this year” and assign probabilities to them.
We can update those probabilities as we gather more evidence—say, spotting a new object in the sky.
And we can reason about risk using exactly the same logic we used to diagnose a cough.

The rules of probability are universal—they help us model and update beliefs whether we're rolling dice, flipping coins, diagnosing disease, or watching the skies.

Gemma 3 270M, the Little Model that Can't

Tue, 16 Sep 2025 09:00:00 -0600

Google’s latest addition to the Gemma family is small, tidy, and full of promise: Gemma 3 270M, a 270-million-parameter model “designed from the ground up for task-specific fine-tuning” and described as a compact foundation for on-device and lightweight deployments.

Read that press release carefully and it’s easy to see why this launch felt exciting: Google highlights a 256K token vocabulary (helpful for rare/specialist terms), strong out-of-the-box instruction following for its size, and production-ready quantization checkpoints so developers can deploy very small, efficient variants.

They even give a concrete energy claim: an INT4-quantized version “used just 0.75% of the battery for 25 conversations” on a Pixel 9 Pro SoC — a headline-friendly detail that reinforces the efficiency story.

On Hugging Face, the model page doubles down on the same positioning: Gemma 3 270M is part of a family meant to be deployable in limited-resource environments (laptops, desktops, local cloud instances) and — importantly — the 1B and 270M sizes are listed with a 32K token context window. That long context claim is one of the model’s most attention-grabbing specs.

Why the marketing sounds good

There’s a clear, defensible niche here:

Cost and speed. Small models are cheaper to train, faster to fine-tune, and can reduce inference costs in production.
Quantization support. QAT (quantization-aware training) checkpoints for INT4 are available, which is a real engineering lever to shrink memory and energy use while keeping acceptable quality for many tasks.
Large context (on paper). A 32K context window on a 270M-parameter model is unusual and, if it worked perfectly, would be useful for document-level tasks without needing a much bigger model.

So far: plausible use cases include specialized classifiers, extraction pipelines, or fleets of tiny experts tuned for narrow jobs — exactly the scenarios Google highlights.

The downside: real usage (my experience)

Theory and PR are one thing; real interactive behavior is another. I tested Gemma 3 270M in a practical setup: I run an “AI Karl Popper” assistant that primes the model with actual Karl Popper quotations and context, then asks questions such as “Tell me about induction.” With a stronger model I get a rich, Popper-like answer:

"Ah, induction! A persistent phantom in the philosophy of science. Many believe it to be the bedrock of scientific progress, the very engine that drives us from observation to theory. Yet, I find myself, perhaps stubbornly, in profound disagreement.

As I state in Realism and the Aim of Science (page 188), by induction, one typically means an argument that progresses from empirical premises—singular or particular observations—to universal conclusions, to a universal theory. This progression is thought to occur either with logical certainty or with probability, as calculated by the calculus of probability. However, the argument against induction, as I've often restated, is simple yet devastating: Many theories, like Newton's, which are thought to arise from induction, are demonstrably inconsistent with their alleged inductive premises."

That’s the kind of grounded, citation-backed, voicey reply I’m aiming for when I feed Popper quotes into the context.

Now — same pipeline, Gemma 3 270M (out of the box, instruction-tuned checkpoint):

"One way to get around all this is to consider the case under consideration from the perspective of one person rather than several people...

The quote below shows what I think is going on right now....and how much more complex and confusing everything has become. My apologies if they have confused me; otherwise, thank you!!!
My notes show that there isn't really any specific problem with 'the concept' [sic] - just its formal formulation ...there would still seem to be many..."

That answer fails in basically the only two things that matter! It doesn’t engage the Popper quotations I provided, and it doesn’t meaningfully even answer the question about induction. In short: Gemma 3 270M, as I used it, ignored the targeted context and produced generic, confused text.

What this implies (and what the sources actually promise)

From the docs and model card you can legitimately take away:

Gemma 3 270M is designed to be a compact, quantizable model for task-specific fine-tuning and efficient deployments. That’s Google’s explicit pitch.
The model does support long context (32K) in specification and the family’s documentation describes deployability on limited-resource environments such as laptops/desktops. But “spec says 32K” ≠ “in every setup it will reliably use 32K of priming text to answer complex, source-grounded questions.”
INT4 QAT checkpoints exist, giving engineers a credible path to very low memory/energy deployments — but practical performance and quality tradeoffs (latency, coherence, grounding) depend heavily on the inference stack, quantization details, and any fine-tuning you perform.

So, the honest takeaway: Google’s engineering and packaging here are meaningful and exciting in an engineering sense — but that doesn’t automatically make a 270M model an out-of-the-box substitute for a larger LLM in tasks that require faithful use of a long context, precise factual grounding, or nuanced reasoning. Indeed, I have my doubts this is useful at all out of the box unless you just want a stupid but chatty mobile phone bot.

Final verdict

I like the direction. Gemma 3 270M is exactly the sort of engineering experiment we need: smaller foundation models that are quantizable, cheap to run, and crafted to be fine-tuned for specific jobs. Google’s blog and the Hugging Face model card show a coherent plan for an ecosystem of small models and the tools to deploy them.

But in practice — at least in my hands and for my “AI Karl Popper” use case — the model was underwhelming as a drop-in replacement for a larger reasoning/model-with-memory setup. If your use case absolutely depends on reliable use of priming context or high-fidelity reasoning, plan to fine-tune heavily on your domain.

Keeping Your AI Chatbot on a Leash: Tiers of Agency

Tue, 09 Sep 2025 09:00:00 -0600

Tiered Querying: The Story of How AI Karl Popper Knows When to Ask for Help

Imagine AI Karl Popper (or your own AI Agent) sitting at his virtual desk, faced with a user’s question. Does he immediately summon the full power of Gemini and ask it to scour the web? Not quite. Instead, he follows a series of ever‑stronger “escalations,” trying the cheapest, fastest trick first, and only calling in the big guns when simpler steps fail. Here’s the heart of that logic:

Tier 1: Raw search results from our Haystack datastore

First AI Karl will try to just work with the user's query directly and see if it gives him a good hit.

retrieved_docs, all_docs = self._doc_pipeline.generate_response(message)
max_score = self.get_max_score(retrieved_docs)

If one of those results earns at least 50 percent confidence, he feels comfortable using them directly.

Tier 2: Let Gemini Rewrite the Query

But if even the best match scores below 0.50, it’s a sign that the user’s phrasing might be muddy. Rather than immediately plunging into expensive LLM research, Karl slips in a polite request to Gemini: “Could you reword this question for me?” With a cleaner search phrase, he tries once more against the same archive and only adopts the new results if they clearly outperform the first batch.

For example, suppose someone asks AI Karl Popper something like this:

My grandma once went to go see Bozo the clown. He told my grandma that he believed in Induction. There was Bozo, with his big shoes and red nose and all of Bozo's friends were getting out of a tiny car. And then he whispered to my grandma "See, you can tell by the first clown that another will follow! So induction is correct! Hume was off base! Honk Honk" And so my grandma told my father who told my mother who told my hairdresser who told my sister who told me that Bozo was right! Induction is correct!

There is a legitimate question hidden there among a lot of ridiculous narrative. There is no way the semantic search is going to find much at all based on the user's query in this case. We offer something like this:

# If nothing scores above 0.50, ask the LLM to rewrite a clearer query
if max_score is not None and max_score < 0.50:
    improved_query = self.ask_llm_for_improved_query(message, gemini_chat_history)
    if improved_query:
        new_docs, _ = self._doc_pipeline.generate_response(improved_query)
        new_max = self.get_max_score(new_docs)
        if new_max > max_score + 0.05:
            retrieved_docs, max_score = new_docs, new_max

Tier 3: Still weak? (max_score < 0.30) Let Gemini itself pick the relevant quotes

Next, if the top score still hasn’t broken 0.30, Karl recognizes that neither the original nor the rewritten query is yielding solid matches. He switches tactics—from refining the question to refining the answers themselves. Here he asks Gemini to look at the low‑scoring candidates and pick out the handful of quotes it deems most relevant. If that succeeds, Karl moves forward with those hand‑picked snippets; if not, he applies a simple filter, discarding any quotes below a modest threshold (0.20 if there are at least three solid contenders, or a lower bar of 0.10 otherwise).

if max_score is not None and max_score < 0.30:
    response_text = self.ask_llm_for_quote_relevance(message, retrieved_docs)
    relevant_ids = [int(x) for x in response_text.split(',') if x.strip().isdigit()]
    ranked_docs = [doc for i, doc in enumerate(retrieved_docs, 1) if i in relevant_ids]
else:
    threshold = 0.20 if sum(d.score >= 0.20 for d in retrieved_docs) >= 3 else 0.10
    ranked_docs = [d for d in retrieved_docs if d.score >= threshold]

Tier 4: If we still lack at least five strong hits or max_score < 0.60,

Finally, even after these steps, there may still not be enough high‑quality material: fewer than five quotes or a top score under 0.60. In that case, Karl waves the white flag on simple retrieval and says, “Alright, Gemini, do your own research.” At this stage, the agent spins up a full ReAct‑style investigation—calling functions, mining external sources (including Wikipedia), and looping until a coherent answer emerges.

#    trigger a full LLM‑powered research sprint
if len(ranked_docs) < 5 or max_score < 0.60:
    research_response, research_docs = self.ask_llm_to_research(message)

By layering these tiers—raw search, query rewriting, quote relevance, and only then full research—AI Karl Popper strikes a balance between efficiency and depth. Most well‑phrased questions end in Tier 1 or Tier 2, saving precious API calls and tokens. Messier queries get gently guided back on track, and only the truly stubborn cases marshal the agent’s full scholarly might. This tiered approach ensures that every question gets the right level of attention, no more and no less.

AI Tutorial: A ReAct Agent Using Gemini and Haystack

Fri, 22 Aug 2025 12:00:00 -0600

Building a ReAct Research Agent with Gemini and Haystack

In a previous post, I walked through how function calling can be used to give large language models the ability to reason and act in a loop — using the ReAct pattern pioneered by this famous paper. In that post, I showed a simple ReAct agent that could chain together "thought → action → observation" steps to work through a question, invoking tools that were functions I had built. In that version, I built the ReAct agent from the ground up which meant I had to parse the results to determine which function to call. But then, in this post (i.e. "Using Gemini Function Calling to Build a Research Agent"), I showed how Google's Gemini had function calling like that built-in. This isn't even the best way to do this! I'll have to show you an easier way in a future post.

In any case, I have released a new version of the Book Search Archive aka "AI Karl Popper" (Mindfire's Open-Source AI software which is really our toy project to work out our open-source stack). It now includes a ReAct based 'research agent' (similar to what we developed in this post: "Using Gemini Function Calling to Build a Research Agent"). I'll explain in a future post why I wanted to do this. In this post, I just want to make people aware of this new version of the code and what has been added.

The ReAct Research Agent — designed specifically to allow "AI Karl Popper" (really just the Book Search Archive with a Karl Popper embedded text database attached) to dig in and come up with its own queries to try to answer a user's question using a combination of document retrieval, function calling, and last-resort fallback to Wikipedia. Yes, that's right, AI Karl Popper can now go to Wikipedia to answer questions. Allowing our pseudo-Karl Popper to update this knowledge beyond the books the real Karl Popper wrote in his lifetime. No bad, eh?

You can view the full code here on GitHub.

What's New in This Agent?

This version of the agent is designed to dig into a custom document store (in my case, a PostgreSQL-backed archive powered by the Haystack framework) and synthesize answers from the most relevant quotes it finds. Only when that fails — and only after six reasoning steps — does it escalate to a fallback query against Wikipedia. (I say after 6 steps, but honestly, AI Karl tends to 'disobey' and call Wikipedia early. Though he's very apologetic about it. No, I'm not making this up. Try him out!)

Key improvements include:

Tool declarations for document and Wikipedia search: Each function is explicitly registered with the Gemini model using Google's function-calling tools. These include search_datastore, search_wikipedia, and answer.
Score-aware document formatting: The quotes pulled from the document store include not just the content, but also metadata such as confidence scores, sources, and retrieval methods. If AI Karl looks up something on Wikipedia a 'document' is created as a reference and the user is given access to what question led to that Wikipedia look up.
Conversation state and flow control: The agent tracks its iteration count, discourages premature Wikipedia access, and gracefully ends when it hits the answer function.

How It Works at a High Level

Here’s a quick summary of the core idea:

Step-by-step reasoning: The agent starts by “thinking aloud” — planning how to approach the question.
Search the datastore first: Using search_datastore, it retrieves top-ranked results from the Haystack-powered database and tries to synthesize an answer.
Fallback to Wikipedia only if needed: If the model still can’t answer the question after several attempts, it uses search_wikipedia to retrieve page content and try again. Try asking AI Karl when Bozo the clown is born and get ready for him to figure out some way to reasonably answer that question even though there are no references to Bozo the clown in Karl Popper's writings and even though it is ambiguous as to what the user intends. (When the character was created? How old the actor would be today?)
Final response with sources: Once it’s ready, the agent calls answer() to return its final result, listing all information sources used.

Here’s an example question you might ask it:

"Is induction valid in some cases, particularly when doing statistics?"

Or...

"When was Bozo the Clown born and how old is he today?"

The agent might begin by planning its approach, then search the document store for any references to, for example, induction, statistical inference, and related concepts. If it finds quotes supporting the idea that induction is a valid form of reasoning in certain empirical domains, it builds an answer from them. Otherwise, it checks Wikipedia for broader philosophical discussion.

Lessons from Gemini's Shifting Tiers

As I noted in this other post, using Gemini effectively now requires a paid API key. Google quietly downgraded the free-tier access earlier this year, making many of these advanced features — especially tool calling — available only via the paid tier. If you're trying to run this agent yourself, make sure to use an authenticated Gemini model (e.g. gemini-1.5-pro or gemini-1.5-flash) with API key access.

What’s Next?

There are several ways I’m looking to improve this:

Better fallback reasoning: Right now, the Wikipedia fallback is a blunt tool. I’d like to add more nuanced decision-making about when to escalate to external sources. And frankly, it's too slow.
Chain-of-thought memory and reasoning: Having the agent remember its past thoughts and decisions more explicitly could help avoid redundant searches.
Cross-query synthesis: Right now, each search is evaluated in isolation. I'd like to experiment with combining results from multiple queries to build richer answers.

If you're interested in trying this out, or building your own research agent, the GitHub repo is a great place to start.

Got questions? Want to contribute? Leave a comment or reach out — I’m always happy to talk shop about AI tooling and research agents.

Using Gemini Function Calling to Build a Research Agent

Fri, 01 Aug 2025 12:00:00 -0600

Imagine you’ve hired a tiny digital research assistant. You ask it a question, it dives first into your local archive of, in my case, Karl Popper essays, then—if it still thirsts for knowledge—dips into Wikipedia, and finally whispers back its findings. All in under a hundred and fifty lines of code. Let’s eavesdrop on how this “research agent” is built, step by step, and then marvel at the real‑world transcript for our inductive statistics question.

You can find all my code in my GitHub repository here.

Rolling Out the Red Carpet

You may need to do some installs. Though, if you've been following along with my posts, you've already got all of these:

pip install google-generativeai wikipedia

Now for some setup:

import wikipedia
import google.generativeai as genai
from doc_retrieval_pipeline import DocRetrievalPipeline
from haystack import Document
from generator_model import get_secret
from llm_message_utils import send_message

# Gemini credentials, plucked safely from disk
gemini_key = get_secret(r'D:\Documents\Secrets\gemini_secret.txt')
genai.configure(api_key=gemini_key)

# Spin up our Popper archive retriever
doc_retriever = DocRetrievalPipeline(
    table_name="popper_archive",
    db_user_name="postgres",
    db_password=get_secret(r'D:\Documents\Secrets\postgres_password.txt'),
    postgres_host="localhost",
    postgres_port=5432,
    db_name="postgres",
    verbose=False,
    llm_top_k=5,
    retriever_top_k_docs=100,
    use_reranker=True,
    embedder_model_name="BAAI/llm-embedder",
)

# Start a fresh Gemini chat session
model = genai.GenerativeModel("gemini-2.0-flash")
chat  = model.start_chat(history=[])

Here, we gently coax Gemini awake, hand it our Postgres‑backed Karl Popper archive, and let it use our API key. Note that we're here using the DocRetrievalPipeline Haystack pipeline we've previously built. (Last encountered here.) We're also going to be using our 'one stop shop' Gemini message sender.

Teaching It to Read

Next, we equip our agent with two “tools”: one for your local Haystack document store and one for Wikipedia. But first, a little formatter to turn hay‑stacked metadata into tidy paragraphs.

def format_doc(doc: Document) -> str:
    lines = []
    for k, v in doc.meta.items():
        if not k.startswith("_"):
            lines.append(f"{k}: {v}")
    lines.append(doc.content)
    return "\n".join(lines)

This helper makes Popper’s chapter titles and page numbers look like a friendly librarian’s margin notes, not raw JSON dumps.

def search_datastore(datastore_query: str) -> Dict[str, str]:
    docs, _ = doc_retriever.generate_response(datastore_query, min_score=0.6)
    if not docs:
        return {"result": "No matching documents found."}
    snippets = [format_doc(d) for d in docs[:3]]
    return {"result": "\n\n".join(snippets)}

def search_wikipedia(wiki_page_search: str, question: str) -> Dict[str, str]:
    try:
        page = wikipedia.page(wiki_page_search, auto_suggest=False)
        return {"result": page.summary}
    except Exception as e:
        return {"result": f"Error fetching Wikipedia: {e}"}

The first function fetches and formats up to three matches from your Popper archive. The second function slips into Wikipedia and grabs the article summary and returns it.

Introducing Our Magic Words

Now we register these functions as tools that Gemini can invoke:

from google.generativeai.types import Tool, FunctionDeclaration

tools = [
    Tool(function_declarations=[
        FunctionDeclaration(
            name="search_datastore",
            description="Query the local Popper archive.",
            parameters={
                "type": "object",
                "properties": {"datastore_query": {"type": "string"}},
                "required": ["datastore_query"],
            },
        ),
        FunctionDeclaration(
            name="search_wikipedia",
            description="Look up a summary on Wikipedia.",
            parameters={
                "type": "object",
                "properties": {
                    "wiki_page_search": {"type": "string"},
                    "question":         {"type": "string"},
                },
                "required": ["wiki_page_search", "question"],
            },
        ),
    ])
]

This tiny registry is the secret handshake between your Python code and Gemini’s function‑calling machinery.

The Heartbeat Loop

Here’s the core loop that sends your question, catches any function_calls, executes them, and feeds the results back—until Gemini has no more tools to invoke:

def run(question: str): prompt = ( "You may call searchdatastore, then searchwikipedia. " "Gather what you need and answer.\n\n" f"Question: {question}" ) response = send_message(chat, prompt, tools=tools)

while True:
    candidate = response.candidates[0]
    did_call = False

    for part in candidate.content.parts:
        # 1) Plain text? Print it.
        if getattr(part, "text", None):
            print(part.text.strip())

        # 2) Function call? Handle it.
        if getattr(part, "function_call", None):
            did_call = True
            func = part.function_call
            name = func.name
            raw_args = func.args or {}
            pretty_args = {}
            for k, v in raw_args.items():
                if hasattr(v, "WhichOneof"):
                    kind = v.WhichOneof("kind")
                    pretty_args[k] = getattr(v, kind)
                else:
                    pretty_args[k] = v

            print(f"\n[Calling {name} with args {pretty_args}]")
            dispatch = {
                "search_datastore": search_datastore,
                "search_wikipedia": search_wikipedia,
            }
            result = dispatch[name](**pretty_args)["result"]
            print(f"[Result]: {result}\n")

            # Feed the result back into Gemini
            response = send_message(chat, result, tools=tools)
            break

    if not did_call:
        # No more function calls → done
        break

Basically we call Gemini and it will give a response. Likely (if things go as intended) it will return with a function call request. We call that function and return the results to Gemini and it will keep calling functions until it's ready to give an answer. Once it stops calling functions it should return a result and that is our researched answer.

Showtime: Watching It Think

Let’s run it with our burning question:

if __name__ == "__main__":
    run("Is induction valid in some cases, particularly when doing statistics?")

The results are pretty amazing for such a small program:

Prompt: You may call search_datastore and then search_wikipedia to gather information. Preferably call both. Then answer the question.

Question: Is induction valid in some cases, particularly when doing statistics?

The agent first makes a search_datastore call:

[Calling search_datastore with args {'datastore_query': 'validity of induction in statistics'}]
[Result]: item_#: 5
page_#: 40
item_id: ch3
file_path: documents\Karl Popper - The Two Fundamental Problems of the Theory of Knowledge-Routledge (2008).epub
book_title: The Two Fundamental Problems of the Theory of Knowledge
paragraph_#: 26
chapter_title: Chapter III THE PROBLEM OF INDUCTION
And yet, all these arguments together still do not suffice to guarantee the validity of the principle of induction.
The inference from our observations as to the validity of the principle of induction in turn represents a generalisation, an inductive inference. And in this generalisation again we (tacitly or explicitly) make an assumption analogous to the earlier assumption formulated in the principle of induction. The only difference is that this time it consists not in the induction of a natural law, but in the induction of a principle of induction.

...

Then it calls search_datastore again with a different question:

[Calling search_datastore with args {'datastore_query': 'inductive reasoning in statistics'}]
[Result]: page_#: 365
file_path: documents\Realism and the Aim of Science -- Karl Popper -- 2017.pdf
book_title: Realism and the Aim of Science -- Karl Popper -- 2017
section_name: CRITICISM OF PROBABILISTIC INDUCTION
Inductive reasoning is in some respects similar to deductive reasoning, although not quite as conclusive. This situation may be explained by assuming that a conclusive (or an almost conclusive) argument corresponds to the probability 1, an inconclusive argument to a lower probability. Thus inductive reasoning will be part of probability logic; and so will some form or other of the simple inductive rule which is the simplest and most elementary form of all inductive reasoning.    

...

Then it tries search_wikipedia:

[Calling search_wikipedia with args {'wiki_page_search': 'Statistical inference', 'question': 'statistical inference'}]
[Result]: Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population.
Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population. In machine learning, the term inference is sometimes used instead to mean "make a prediction, by evaluating an already trained model"; in this context inferring properties of the model is referred to as training or learning (rather than inference), and using a model for prediction is referred to as inference (instead of prediction); see also predictive inference.
 Have you called both functions yet? If not, please do so before answering.

With all that information researched, here is the research agents final response to me:

Response:
Based on the information gathered:

*   **Induction is not deductively valid.** Karl Popper emphasizes that induction is never valid in the sense of deductive reasoning. In cases where all instances have been observed, it essentially becomes deduction.
*   **Induction is used in statistics.** Statistical inference uses data analysis to infer properties of an underlying probability distribution. It infers properties of a population by testing hypotheses and deriving estimates.
*   **Induction is related to probability.** Some sources suggest inductive reasoning can be related to probability logic, where conclusive arguments have a probability of 1, and inconclusive arguments have lower probabilities.
*   **Alternative perspectives exist.** Some argue that inductive reasoning has its own standards and can be considered "inductively valid" if it conforms to those standards. However, this doesn't make it deductively valid.

In summary, while induction is not deductively valid, it is used in statistics to make inferences about populations based on observed data. The validity of induction in statistics relies on probabilistic reasoning and specific inductive standards rather than deductive certainty.

That's a pretty impressive result!

This example gives you the basic idea of both how to do function calling for Google's Gemini and also how to create a very basic Large Language Model agent to do research for you.

Writing a Robust “Do‑It‑All” Gemini API Caller in Python

Thu, 24 Jul 2025 12:00:00 -0600

In our last post, we talked about how Google killed the rate limits on their free tier. How can we best deal with that problem programmatically? I wrote a utility function that wraps calls to the Gemini API and does all the following:

Send either a plain “generate content” call or a full chat message
Catch and parse rate‑limit errors (ResourceExhausted)
Back off for the exact number of seconds Google suggests

However, even with this functionality, you can get yourself into an infinite loop of retries because Google will lie to you about how long you need to wait before you can try again. (Presumably due to going over your -- now very small -- daily allotment, though I'm somewhat doubtful that they don't just rate limit you before you've reached your daily allotment.)

The "send_message" Utility

Drop llm_message_utils.py into your project (from BookSearchArchive):

from google.api_core.exceptions import ResourceExhausted
from google.generativeai import ChatSession, GenerativeModel
from google.generativeai.types.generation_types import GenerationConfig, GenerateContentResponse
from typing import Any, List, Union
import time
import re

def _extract_retry_seconds(exc: ResourceExhausted, default: int = 15) -> int:
    """
    Parses "retry_delay { seconds: N }" from exc.details.
    Returns N if found, else returns default.
    """
    try:
        details = str(getattr(exc, "details", ""))
        match = re.search(r'retry_delay\s*{\s*seconds:\s*(\d+)', details)
        if match:
            return int(match.group(1))
    except Exception:
        pass
    return default

def send_message(
    model: Union[ChatSession, GenerativeModel],
    message: str,
    tools: List = None,
    stream: bool = False,
    config: GenerationConfig = None,
    **generation_kwargs: Any
) -> Union[GenerateContentResponse, str]:
    # Build GenerationConfig from kwargs if needed
    if config is None and generation_kwargs:
        config = GenerationConfig(**generation_kwargs)

    try:
        if isinstance(model, ChatSession):
            # Full chat support (streaming, tools)
            return model.send_message(
                message,
                generation_config=config,
                tools=tools,
                stream=stream
            )
        else:
            # Single-shot content generation
            response = model.generate_content(
                contents=message,
                generation_config=config,
                tools=tools,
                stream=stream
            )
            return getattr(response, "text", None) or "[No response text]"

    except ResourceExhausted as e:
        delay = _extract_retry_seconds(e, default=15)
        print(f"Rate limit hit. Backing off for {delay} seconds…")
        time.sleep(delay)
        # WARNING: If you’ve hit your daily quota, this will loop forever!
        return send_message(
            model,
            message,
            tools=tools,
            stream=stream,
            config=config,
            **generation_kwargs
        )

How it Works

Let's start with the _extract_retry_seconds method. This method takes a ResourceExhausted error and gets the "retry_delay" value, which is supposed to be the number of seconds before you can retry again. As mentioned in our last post, this may do you no good if you reached the daily limit.

The send_message method does the real work. At a minimum, you need to pass a model and a message/prompt. The 'model' can be either a GenerativeModel or a ChatSession taken from a GenerativeModel. ChatSessions have a lot of extra functionality like tracking history. I try to abstract away some of the pain here by allowing you to pass either. I'm assuming that if you pass a ChatSession you want to talk to the Large Language Model (LLM) in a chat session format, otherwise you just want to generate come content based on the prompt.

If this is a chat session, I call:

model.send_message(
                    message,
                    generation_config=config,
                    tools=tools,
                    stream=stream
                )

Note that I allow you to pass in configuration arguments, a list of Tools, and a boolean value for streaming or not.

If you just want to generate content, I instead call:

response = model.generate_content(
    contents=message,
    generation_config=config,
    tools=tools,
    stream=stream
    )

return getattr(response, "text", None) or "[No response text]"

Note that if this is a chat session we're returning a GenerateContentResponse so that it comes with the chat history. But if we're generating content, we just return a regular string.

We wrap all that into a try exception block and trap ResourceExhausted errors. From there we just call the _extract_retry_seconds helper function and then sleep for the number of seconds they told us to. If they didn't give us a number, we default to 15 seconds.

except ResourceExhausted as e:
  delay = _extract_retry_seconds(e, default=15)
    print(f"Rate limit hit. Backing off for {delay} seconds…")
    time.sleep(delay)
    # WARNING: If you’ve hit your daily quota, this will loop forever!
    return send_message(
        model,
        message,
        tools=tools,
        stream=stream,
        config=config,
        **generation_kwargs
      )

I now can call all LLM calls via this utility function and I'm getting retries for free. We could obviously add a lot more here like exponential backoff or smarter checking for how long to wait.

We'll be using this little utility in BookSearchArchive for all our calls in preparation for the ReAct based research agent we're going to build next to allow our agent to do its own queries to research and find an answer to a user's query.

Google Quietly Ruins Gemini’s Free Tier

Thu, 17 Jul 2025 09:00:00 -0600

Google has quietly chopped the Gemini API free tier down to size—and now it’s about as useful as a screen door on a submarine. Once upon a time, you could fire off 1,500 requests per day, but now Gemini 2.0 Flash free tier is capped at 200 requests per day (Google Rate Limits “Free Tier” table; previously 1,500/day according to Zapier: "Free up to 15 requests/min; 1M tokens/min; 1,500 requests/day")

Meanwhile, if you switch to Gemma—which on paper allows 30 requests per minute and a whopping 14,400 requests per day (Google Rate Limits “Gemma 3 & 3n” row)—you’ll still hit a ResourceExhausted error long before you approach those limits (Google Support).

I doubt I’m even burning through as many requests as Google claims. For example, my test script throws “ResourceExhausted” after just a handful of calls—despite staying well under the documented RPM and TPM caps. That means prototyping or smoke‑testing your app without buying tokens is now effectively impossible. Want to validate your UX? Better hand over your credit card. Moreover, you'll hit this Gemma limit even if you have a paid tier!

For anyone chasing budget‑friendly LLM options, it might be time to head back to Hugging Face or other open‑source alternatives. Because if Google won’t let you play in the sandbox for free, you’ll have to build your sandcastle elsewhere.

Other Related Links:

Mindfire Technology

Adventures in LangChain's "Quick Start Tutorial" (Using Ollama)

Weekend Warrior Project: AI Patent Checking

How It Works: The Tech Behind the Magic

1. Gradio for a Clean, Instant UI

2. Local LLMs with Ollama

3. DSPy: Structured AI Orchestration

4. Multi-Stage Patent Retrieval

5. FlashRank Reranking

6. Full Patent Document Retrieval

7. Semantic Reranking with LLMs

The Result

AI Patent Search Agent Pipeline

Why Mindfire Can Help

From Prompts to Applications: A Beginner’s Introduction to LangChain

What LangChain Actually Is

Installing LangChain

A Quick Start Example

When LangChain Shines

When to Be Careful

LangChain in the Broader Ecosystem

Final Thoughts

AI: Bubble or Breakthrough? A Look at the Risks and the Rewards

The Pessimistic View: AI as “Funny Money” and Fragile Economics

The Optimistic Take: Jeff Bezos on the “Industrial Bubble”

So What Should We Make of It?

🚨 The Bubble Side

🌟 The Breakthrough Side

So I Should Be Afraid of the AI Bubble?

Conclusion: Bubble or Breakthrough?

Building a Local DeepSeek R1 Chatbot with Chainlit

What You'll Need

The Complete Code

Breaking Down the Code

Chainlit Event Hooks

LaTeX Conversion Helper

Thinking Step

Streaming with Ollama

Streaming Back to Chainlit

Welcome Message

Error Handling

Running the Application

Key Features

Concerns

Conclusion

Building a Local DeepSeek R1 Chatbot with Streamlit and Ollama

What You'll Need

The Complete Code

Breaking Down the Code

Page Configuration

LaTeX Conversion Function

Initializing Chat History

Displaying Chat History

Handling User Input

Streaming the Response

Processing the Stream

Finalizing the Display

Saving to History

Running the Application

Key Features

Conclusion

How DSPy Optimizes Prompts

A Poorly Defined Prompt

Running the Optimizer

Conclusion

How DSPy Builds Prompts

How Classify Works

How it Works

Conclusion

DSPy: A Powerful (But Sometimes Dangerous) Prompting Tool

Writing a "Classify" Function

Explanation of the Classify function, line by line:

Running the Code

How Prompts Are Built

Conclusion

DSPy: Prompt your LLM Like It's Code

Why DSPy Instead of Prompt Building?

Getting Started with DSPy

Understanding the ChainOfThought Signature

Conclusion

Explanation of the `Classify` function, line by line:

Google Gemini 3 Makes a Huge Leap on the ARC‑AGI Benchmark