Understanding Open-Source Large Language Models: From Hugging Face to Ollama

7 Jul, 2026
llm open-source ollama huggingface
6 min read

A detailed summary for learning and teaching LLM

1. What do we actually download from Hugging Face?

One of the biggest misconceptions is that downloading a Large Language Model (LLM) means downloading its training data.

This is NOT true.

When you download an open-source model from Hugging Face, you are primarily downloading a trained neural network, not the data that trained it.

Think of it like this:

Training data = millions or billions of books the student has read
Model weights = everything the student has learned
Source code = the student’s brain architecture
Tokenizer = the student’s dictionary

The books are not included.

Instead, the downloaded model contains the “knowledge” compressed into billions of numerical parameters.

2. What files are inside a Hugging Face model?

A model such as Llama-2-7B usually contains files like these:

config.json
generation_config.json

pytorch_model-00001-of-00002.bin
pytorch_model-00002-of-00002.bin
(or model.safetensors)

tokenizer.model
tokenizer.json
tokenizer_config.json
special_tokens_map.json

README.md
LICENSE

Each file has a different purpose.

config.json

This defines the architecture of the neural network.

For example:

number of Transformer layers
hidden dimension
number of attention heads
vocabulary size

It is the blueprint of the model.

Model weights

Example:

pytorch_model.bin

model.safetensors

These are the trained parameters.

Inside are billions of floating-point numbers.

For Llama-2-7B:

around 7 billion parameters
approximately 14 GB in FP16 format

These numbers represent everything the model learned during training.

Tokenizer

Files such as

tokenizer.model
tokenizer.json

convert text into numbers.

Example:

Hello world

becomes

[15043, 3186]

The model never understands English directly.

It only understands token IDs.

Generation configuration

generation_config.json

Stores default generation settings such as

temperature
top-p
top-k
max tokens

README and LICENSE

These are documentation.

They are not loaded into memory during inference.

3. How do these files work together?

The complete pipeline looks like this.

User types text
        │
        ▼
Tokenizer
(text → token IDs)
        │
        ▼
Transformer model
(config + weights)
        │
        ▼
Predicted token IDs
        │
        ▼
Tokenizer
(token IDs → text)
        │
        ▼
Final answer

4. Mapping files to Python objects

When using the Transformers library:

config.json
        │
        ▼
LlamaConfig object

weights.bin
        │
        ▼
state_dict
        │
        ▼
LlamaForCausalLM

tokenizer.model
        │
        ▼
SentencePiece tokenizer

generation_config.json
        │
        ▼
GenerationConfig

At runtime, these become actual Python objects.

5. What happens inside `from_pretrained()`?

When we execute

model = AutoModelForCausalLM.from_pretrained(...)

the library performs several steps automatically.

Step 1

Locate the model

local folder
or

Hugging Face Hub

Step 2

Read

config.json

Create

LlamaConfig

Step 3

Create an empty neural network

LlamaForCausalLM(config)

At this moment:

layers exist
weights are empty

Step 4

Load

pytorch_model.bin

Convert it into

state_dict

Step 5

Copy every tensor into the corresponding layer.

For example

layer0.attention.q_proj.weight

is copied into

Layer 0
↓
Attention
↓
Query projection

Step 6

Return the fully initialized model.

6. What happens during text generation?

Suppose the prompt is

Hello

The generation loop looks like this.

Prompt
   │
   ▼
Tokenizer
   │
   ▼
Token IDs
   │
   ▼
Forward pass
   │
   ▼
Probability of next token
   │
   ▼
Choose one token
   │
   ▼
Append token
   │
   ▼
Repeat

The model predicts one token at a time.

It never writes the whole sentence in one computation.

7. Hugging Face vs Ollama

These two systems play very different roles.

Hugging Face

Provides

model files
tokenizer
configuration

You are responsible for

Python
PyTorch
Transformers
CUDA
inference code

It is flexible but requires programming knowledge.

Ollama

Ollama is more like a runtime platform.

It provides

model management
inference engine
API server
command-line interface

You simply run

ollama run llama3

without writing Python.

8. What does Ollama actually do?

Ollama has several important responsibilities.

Model management

Commands such as

ollama pull
ollama list
ollama rm

download and organize models.

Runtime engine

Internally it uses technology based largely on llama.cpp.

It performs

matrix multiplication
memory management
GPU/CPU scheduling

API server

Ollama exposes a local REST API.

Applications simply send

Prompt

and receive

Generated text

User interface

Instead of writing Python,

you simply type

ollama run llama3

9. Hugging Face model structure vs Ollama model structure

Hugging Face

config.json

weights.bin

tokenizer.model

tokenizer.json

generation_config.json

Many files.

Ollama

Usually only

model.gguf

plus some metadata.

Everything is packaged together.

10. What is GGUF?

GGUF is a model format designed primarily for llama.cpp.

Unlike Hugging Face,

GGUF stores

architecture
tokenizer
vocabulary
weights
metadata

inside one file.

Think of it as a self-contained package.

11. Why use GGUF?

Advantages:

smaller size
quantized
faster loading
lower RAM usage
works well on CPUs
easier deployment

Instead of 14 GB,

a 7B model may become only

4 GB

after 4-bit quantization.

12. How does a Hugging Face model become an Ollama model?

The conversion pipeline is

Hugging Face

config.json

weights.bin

tokenizer.model
        │
        ▼
Download
        │
        ▼
Conversion tool
(llama.cpp)
        │
        ▼
GGUF
        │
        ▼
Ollama
        │
        ▼
Run

The conversion process

reads Hugging Face files
packs them into GGUF
optionally quantizes the weights

13. Practical workflow

A typical workflow looks like this.

Step 1

Install tools

git-lfs

llama.cpp

Ollama

Step 2

Download the model

git clone

huggingface-cli download

Step 3

Convert

weights.bin

into

model.gguf

Step 4

Optionally quantize

For example

Q4_K_M

to reduce memory usage.

Step 5

Create a Modelfile

FROM model.gguf

Step 6

Import into Ollama

ollama create

Step 7

Run

ollama run

curl http://localhost:11434/api/generate

14. The big picture

The entire ecosystem can be summarized as follows.

                 Training
                     │
                     ▼
      Massive training dataset
                     │
                     ▼
      Billions of learned parameters
                     │
                     ▼
         Hugging Face Repository
                     │
                     ▼
      Download config + tokenizer + weights
                     │
                     ▼
          Convert to GGUF (optional)
                     │
                     ▼
              Quantization
                     │
                     ▼
             Ollama Runtime
                     │
                     ▼
        Tokenizer → Transformer → Output
                     │
                     ▼
          Human-readable response

15. Key concepts to remember

Training data is almost never included with an open-source model. The model contains learned parameters, not the original dataset.
The model weights are the “brain” of the AI. They store everything the model learned during training.
The configuration defines the neural network architecture, while the weights fill that architecture with learned knowledge.
The tokenizer is essential. It converts human language into token IDs and converts generated token IDs back into text.
from_pretrained() automates model loading by reading the configuration, creating the network, loading the weights, and returning a ready-to-use model.
Text generation is iterative. The model predicts one token at a time until it reaches an end-of-sequence token or a maximum length.
Hugging Face is primarily a model repository and ecosystem, whereas Ollama is a local inference runtime and model management platform.
GGUF packages everything into a single optimized file, making deployment simpler and enabling efficient CPU or low-memory inference through llama.cpp-based runtimes.

This progression—from understanding what a model contains, to how it is loaded, how inference works, and how deployment differs between Hugging Face and Ollama—provides a strong conceptual foundation for anyone beginning to learn modern large language models. It emphasizes not just how to use these systems, but why they are designed the way they are.

1. What do we actually download from Hugging Face?

2. What files are inside a Hugging Face model?

config.json

Model weights

Tokenizer

Generation configuration

README and LICENSE

3. How do these files work together?

4. Mapping files to Python objects

5. What happens inside from_pretrained()?

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

6. What happens during text generation?

7. Hugging Face vs Ollama

Hugging Face

Ollama

8. What does Ollama actually do?

Model management

Runtime engine

API server

User interface

9. Hugging Face model structure vs Ollama model structure

Hugging Face

Ollama

10. What is GGUF?

11. Why use GGUF?

12. How does a Hugging Face model become an Ollama model?

13. Practical workflow

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

Step 7

14. The big picture

15. Key concepts to remember

5. What happens inside `from_pretrained()`?