Ollama

Overview

Ollama is an open-source tool that lets you run large language models locally on your own hardware. It now supports the latest cutting-edge models including Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and many others. Ollama makes it easy to download, run, and customize these models through a simple API, allowing for seamless integration into your applications and workflows.

System Requirements

• CPU: 4+ cores
• RAM: 8GB+ (minimum for 7B models), 16GB+ (for 13B models), 32GB+ (for 33B models)
• GPU: Optional, NVIDIA GPU with 8GB+ VRAM recommended for better performance
• Storage: 10GB+ depending on the models you install

Installation Guide

Prerequisites

• Basic knowledge of command line interfaces
• Git installed on your system (for source installation)
• Docker and Docker Compose (recommended for easy setup)
• NVIDIA GPU with appropriate drivers installed (recommended for better performance)

Option 1: Direct Installation (Recommended)

macOS

brew install ollama

Linux (Ubuntu 20.04+, Debian 10+, RHEL 8+)

curl -fsSL https://ollama.com/install.sh | sh

Windows

Download the installer from the Ollama website

Option 2: Docker Installation

1. Pull the official Ollama Docker image:

   docker pull ollama/ollama
   

2. Run the container:

   docker run -d -p 11434:11434 -v ollama:/root/.ollama ollama/ollama
   

Option 3: Build from Source

1. Clone the repository:

   git clone https://github.com/ollama/ollama.git
   

2. Navigate to the project directory:

   cd ollama
   

3. Build and install:

   go build
   

4. Run Ollama:

   ollama serve
   

Practical Exercise: Getting Started with Ollama

Now that you have Ollama installed, let’s walk through a simple exercise to help you get familiar with the basics.

Step 1: Download a Model

First, let’s download one of the latest models:

ollama pull llama3.2

Step 2: Chat with the Model

Now, let’s start a conversation with the model:

ollama run llama3.2

You’ll be placed in an interactive chat where you can ask questions and get responses.

Step 3: Using the Model with Images (Multimodal)

If you want to use a multimodal model that can process images:

ollama pull llava
ollama run llava "What's in this image?" /path/to/your/image.jpg

Step 4: Customizing a Model

You can create a customized version of a model with a Modelfile:

1. Create a file named Modelfile with the following content:

FROM llama3.2
PARAMETER temperature 1
SYSTEM """
You are a helpful coding assistant who specializes in explaining complex programming concepts.
"""

2. Create your custom model:

ollama create coding-assistant -f Modelfile

3. Run your custom model:

ollama run coding-assistant

Step 5: Using the API

Ollama provides a REST API that you can use to integrate models into your applications:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Why is the sky blue?"
}'

Step 6: Setting Context Length

You can customize the context length using environment variables:

OLLAMA_CONTEXT_LENGTH=8192 ollama serve

Resources

Official Documentation and Links

• Ollama Website
• GitHub Repository
• Model Library
• Modelfile Documentation
• API Documentation

Community and Extensions

• Python Client
• GitHub Topics: Ollama - Find community tools and extensions
• GitHub Issues - Get help and report bugs

Ollama Ecosystem

Ollama has a growing ecosystem of tools and applications:

• Web interfaces for chatting with Ollama models
• VSCode extensions for code completion and assistance
• Libraries for various programming languages
• RAG (Retrieval-Augmented Generation) implementations
• Desktop applications with Ollama integration

For more information and the latest updates, visit the Ollama GitHub repository.