Ollama
Ollama lets you download, run and integrate AI models on your own computer, with cloud options for larger workloads.
Why it caught my attention
Running AI models no longer has to mean relying entirely on an external provider. Ollama makes it relatively straightforward to experiment with models on your own hardware, while offering cloud access when local resources are insufficient. That flexibility creates interesting options for developers and IT teams.
What problem does this actually solve?
Ollama could be worth considering if you recognise any of these challenges.
Sensitive data leaves your environment
Run suitable models locally so prompts and responses need not be sent to an external inference service.
Cloud inference costs can accumulate
Use local hardware for repeated AI tasks without paying per-request cloud inference charges.
Trying different models takes effort
Download and switch between supported models using a consistent interface instead of configuring separate inference systems.
Applications need accessible AI capabilities
Connect applications and developer tools to models through APIs, without building an inference engine yourself.
Is this a good fit for you?
A quick way to see if Ollama is relevant for your situation.
- You want to experiment with AI models on your own computer.
- Your applications need access to locally hosted AI capabilities.
- You prefer keeping sensitive prompts within infrastructure you control.
- You want to compare different models without changing your entire application.
- You need optional cloud access for models exceeding local hardware capacity.
- You only need a fully managed AI chatbot with no technical setup.
- Your hardware cannot accommodate suitable models and cloud execution is unacceptable.
How does it work?
In practice, the workflow looks something like this:
Download Ollama for a supported operating system and prepare the local environment where you want models to run.
Browse available models and select one based on your task, hardware capacity and preferred capabilities.
Download and start a local model, then submit prompts to check its responses and performance on your machine.
Use Ollama's API or supported integrations to send requests from your software, scripts or development tools.
Select a hosted model when your hardware cannot run a suitable local alternative, subject to account and usage requirements.
What else should I know?
A few important things to be aware of before you investigate further.
Local hardware determines performance
Model size, available memory and GPU capabilities affect which models run comfortably and how quickly they respond.
Local and cloud privacy differ
Local inference avoids external processing, while cloud models send requests to Ollama's hosted infrastructure under its privacy policies.
Cloud usage has separate charges
Local execution has no Ollama inference fee; hosted models use token-based pricing and may require purchased credits.
Model licences need individual checks
Models distributed through Ollama can have different licences, restrictions and permitted uses; review each model before commercial deployment.
Think Ollama could fit your environment?
Visit the Ollama website to find out more, request a demo or speak to their team.
Something you'd like to know about Ollama?
Is there something I haven't covered in my research? Send me your question and I'll look into it and reply personally by email.