Integrate Ollama with ELK Stack for AI-Driven Logs

techhadoop ai, ELK, llm

Here is how to set up Ollama on your computer to run large language models locally.

1. Download and Install

  • Mac: Download the zip file from the Ollama website, unzip it, and drag Ollama to your Applications folder.
  • Windows: Download the Windows installer (OllamaSetup.exe) from the website and run it.
  • Linux: Run the following command in your terminal:Bashcurl -fsSL https://ollama.com/install.sh | sh

2. Verify the Installation

Open your terminal (Mac/Linux) or Command Prompt/PowerShell (Windows) and type:

Bash

ollama --version

If it returns a version number, Ollama is running.

3. Run a Model

To download and start chatting with a model, use the ollama run command followed by the model name. For example, to run Llama 3:

Bash

ollama run llama3

The first time you run this, Ollama will download the model weights (usually a few gigabytes), so it might take a few minutes depending on your internet speed.

Once the download finishes, you will see a >>> prompt. You can now type your questions and chat with the model directly in your terminal.

4. Useful Commands

  • /bye : Type this inside the chat prompt to exit the model.
  • ollama list : Shows all the models you have downloaded on your machine.
  • ollama rm <model_name> : Deletes a downloaded model to free up space.
  • ollama pull <model_name> : Downloads or updates a model without opening the chat interface.

Popular Models to Try

  • ollama run mistral (Great all-rounder)
  • ollama run phi3 (Lightweight and fast for lower-spec machines)
  • ollama run codegemma (Tuned for coding tasks)

Integrating Ollama with the ELK Stack (Elasticsearch, Logstash, Kibana) allows you to bring the power of localized Large Language Models directly to your log telemetry. Instead of manually writing complex regex or staring at lines of stack traces, you can use Ollama to explain cryptic error codes, detect anomalous behavior, and summarize massive log spikes.

There are two primary architectural patterns to achieve this:

  • Pattern A (Reactive Pipeline): Passing logs through a Python script or Logstash webhook into Ollama before indexing them into Elasticsearch.
  • Pattern B (Elastic Inference API): Using Elasticsearch’s native AI capabilities to query Ollama directly as an external inference service.

Below is the implementation guide for Pattern A, which is the most reliable, customizable, and widely used method for custom AI log processing.

The Architecture

  1. Logstash / Filebeat collects the logs.
  2. An Intermediate Processor (Logstash Webhook or a Python Worker) intercepts the logs, calls Ollama’s local HTTP API, and gets an AI summary or anomaly score.
  3. The enriched log (Original data + AI Explanation) is indexed into Elasticsearch.
  4. You visualize the AI-generated insights in Kibana.

Step 1: Prepare your Local Ollama Model

You want a model that is fast and skilled at code or technical troubleshooting. Mistral, Llama3, or Phi3 are highly recommended.

  1. Start Ollama and pull your model:Bashollama pull mistral
  2. Verify that Ollama’s local API is listening on port 11434 by querying it via curl:Bashcurl http://localhost:11434/api/generate -d '{ "model": "mistral", "prompt": "What does a 500 Internal Server Error mean?", "stream": false }'

Step 2: Set up the Ingestion and AI Enrichment

Because logs stream continuously and can overwhelm an LLM, you typically want to filter logs so only Errors or Warnings get sent to Ollama.

You can accomplish this using Logstash’s HTTP Filter plugin to dynamically call Ollama during ingestion.

Logstash Configuration (logstash.conf)

Ruby

input {
  # Your standard log input (e.g., Filebeat, Syslog, Beats)
  beats {
    port => 5044
  }
}
filter {
  # 1. Filter out info logs; only process actual errors through the LLM to save CPU/GPU cycles
  if [log][level] == "error" or [log][level] == "critical" {
    
    # 2. Use the HTTP filter to hit Ollama's local API
    http {
      url => "http://localhost:11434/api/generate"
      method => "post"
      
      # Craft a robust system prompt + your raw log message
      body => '{"model": "mistral", "prompt": "Analyze this system log. Provide a 1-sentence root cause explanation and a 1-sentence fix. Log: %{[message]}", "stream": false}'
      
      headers => {
        "Content-Type" => "application/json"
      }
      
      # Target field where Ollama's JSON response will land
      target_body => "[@metadata][ollama_response]"
    }
    # 3. Parse out the textual string from Ollama's JSON structure
    if [@metadata][ollama_response] {
      json {
        source => "[@metadata][ollama_response]"
        target => "[ollama]"
      }
      
      # Rename the output to a clean root-level field
      mutate {
        add_field => { "ai_analysis" => "%{[ollama][response]}" }
      }
    }
  }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "ollama-enriched-logs-%{+YYYY.MM.dd}"
    # user => "elastic"      # Uncomment if security is enabled
    # password => "changeme" # Uncomment if security is enabled
  }
}

Step 3: View the Results in Kibana

Once Logstash restarts with the updated configuration, look at your incoming indices.

  1. Go to Kibana -> Management -> Data Views and create a view for ollama-enriched-logs-*.
  2. Navigate to Discover.
  3. Add the field message (your raw system error) alongside your brand new custom field: ai_analysis.
Example of what you’ll see in Kibana:
  • message: [2026-07-08 14:02:11] FATAL: connection back-off failure, system.db.Pool empty. Max capacity 50 reached.
  • ai_analysis: Root Cause: The application has exhausted its database connection pool because connections aren’t being closed properly. Fix: Increase the max_connections parameter in your database configuration or audit your code for connection leaks.

Advanced Architecture Strategy: Vector Search (RAG)

If you have massive logs and don’t want Logstash to trigger Ollama sequentially for every single error, you should shift to a RAG (Retrieval-Augmented Generation) workflow using Elasticsearch as a Vector Database.

  1. You use an embedding model (like ollama pull nomic-embed-text) to convert error logs into vectors.
  2. Store these vectors in Elasticsearch natively using the dense_vector field type.
  3. When a new system anomaly breaks out, write a Python/LangChain script to fetch similar past errors from Elasticsearch, bundle them together, and feed them into Ollama. This lets Ollama explain the current issue using historical context unique to your company’s network stack.

Leave a Reply