What it means to run a language model on the company network, and what managers should consider before starting a pilot.
Summarising a report, preparing a first draft, organising notes or reviewing source code are tasks where a language model may save time. When the material is confidential, however, the question is not only which model to choose. It also matters where the text is processed, who can use the service and what controls govern the project.
One option is to run a model on an internal server and make it available to computers on the company network. Ollama helps manage and run language models; employees can use the service through a chat program or an internal web interface. The model runs on the server while people continue working from their PCs.
This setup can be useful when an organisation wants to explore generative AI while keeping processing within its own infrastructure. Installing the software alone does not automatically make an environment private, secure or offline. The overall design and operating rules determine how data is handled.
The video outlines the basic setup: an internal server runs the model, and computers on the company network send it requests. The transcript of the on-screen captions is available below.
Read the video transcript
- Introduction. Riccardo Corsanici, IT Project Manager. Take Two brings together IT consulting, training and Open Source.
- Architecture. An LLM can run on a company server. Computers inside the company access it over the local network.
- Requests. Ollama and the model run on the server; PCs send requests and receive responses over the LAN.
- Processing. The client sends a prompt to Ollama. The model handles it on the server and returns the answer to the PC.
- Access. Set the server’s internal address and model name. Restrict access to authorised devices.
- Offline use. Prepare software, the model and dependencies before disconnecting from the Internet. Configuration flags alone do not block every connection.
- Evaluation. Test quality, speed, memory and access in a controlled pilot. Then decide whether to expand.
What a local model offers
With a local service, the organisation can choose the model, the hardware and the people allowed to use it. It can also decide which interfaces to adopt and how to manage updates and access. These choices give the organisation more control, while also bringing work that would otherwise be handled in part by a managed-service provider.
The model does not become an expert in the company’s processes. It can produce useful drafts and summaries, but it can also make mistakes, leave out details or present an uncertain result as fact. Responses need review, especially when they affect decisions, customers, employees or regulatory obligations. A sensible first use is a limited task with results that a person can check.
What “local” really means
In the simplest setup, the model and the software running it are on a company server; PCs send requests over the internal network. Employees see a chat interface while the server provides the model and computing resources.
This describes the path taken by a request, not every feature of the programs involved. A client may offer web search, synchronisation or cloud services; another application may send data to external systems. A local installation can still depend on Internet connections unless it is configured and tested carefully.
“Local” also does not mean that the model can automatically read company files, procedures or archives. Access to documents requires a separate integration and careful permission management, so each person sees only the information they are entitled to access.
Decisions before a pilot
For a manager, the first decision is not the brand of graphics card. It is to identify a specific task and agree on what a useful result looks like. For example, a team could turn non-confidential reports into draft procedures and assess accuracy, time saved and the amount of editing required.
Performance depends on the model, the server, the length of the input and the number of simultaneous requests. A test with one person may feel fast and become slow when more employees use it. Before buying hardware or expanding the service, measure the workload with realistic examples and representative users.
The full cost includes more than the server: electricity, maintenance, updates, security, user support and the time of the people responsible for the service. A local model may suit particular needs, but it is not automatically cheaper than an external service. Compare options against the tasks, expected usage and responsibilities the organisation is prepared to take on.
Security and responsibility
Only authorised devices and people should be able to reach the service. Ollama’s direct local API does not require authentication, and its HTTP connection does not encrypt traffic; it should not be exposed to the company network without safeguards. The official authentication documentation describes the local API behaviour. Access rules, network segmentation and an architecture approved by IT and security teams are needed.
It is also useful to set clear rules for employees: which data they may enter, which results must be checked and who is accountable for the final use. Data minimisation remains important even when processing happens on an internal server. The initial pilot can use fictional or already-authorised material.
Working without Internet access requires additional preparation. Software, the model, drivers and dependencies must be available in advance and compatible; the organisation also needs a process for checking and distributing security updates. Disconnecting a network cable is not a replacement for keeping the environment current and controlled.
A measurable pilot
A useful pilot has a small scope and agreed criteria. Before it starts, define the task, the users involved and the data that may be used. During the test, assess response quality, speed, service continuity, simultaneous requests and the effort required to correct mistakes.
At the end, the team can decide whether the model is useful for that task, whether more resources are needed or whether to stop. If the result is positive, wider use requires planning for capacity, support, training, updates and accountability. A promising lab result is a starting point, not proof that the service is ready for the whole organisation.
Ollama can make a model available on the internal network and give an organisation more control over its runtime environment. It makes sense when a concrete use case justifies the resources and governance involved. The most useful next step is a limited pilot with clear criteria and a joint review by IT, security and the process owners.