Local AI or cloud AI: cost, data protection and effort compared

What sets local AI and cloud AI apart

With cloud AI such as ChatGPT, Microsoft Copilot or Google Gemini, the language models run in the vendor's data centers. Every query and every uploaded document is transferred there and processed there.

With local AI, also called on-premises AI, the language model runs on your own hardware, usually an open model on an AI server. Queries and documents stay in your own network. In between sits a third option: AI operated by a German provider in German data centers, without a US cloud.

Cost: per-user licences or your own hardware

Cloud AI for organisations is usually billed per user and month, so cost grows with every seat. A worked example with an assumed licence of 30 euros per user and month: 20 users cost 600 euros a month, 50 users 1,500 euros.

Local AI mainly costs hardware and setup. Instead of a licence per head you pay a monthly rate for the hardware, set by the computing power you need: when more people work with the AI at the same time, it takes more or stronger hardware. The Sinabox starts at 500 euros a month, as the Standard model and as the 19-inch rack, both excluding VAT with a 12-month minimum term. How many concurrent users one device supports depends on the model and usage and is sized together up front. So cost rises in steps with the hardware, not with every single seat.

Cloud AI licences do not include the cost of data protection assessments, contracts and documentation. For professionals bound by secrecy and for authorities this effort can be considerable.

Data protection and confidentiality

Cloud AI requires a data processing agreement, an assessment of possible third-country transfers and clear rules on whether inputs are stored or used for training. With US vendors the CLOUD Act adds to this, as it can give US authorities access to data even in European data centers. Law firms, tax advisers and doctors in Germany are also bound by Section 203 StGB.

With local AI the data never leaves the building. There is no third-country transfer, no external provider with access to content and no question about training data. Data protection documentation becomes much shorter.

Performance: what local models can do

The largest cloud models are often still ahead of open models on general knowledge, very long texts and creative tasks. For typical office work, however, a local model is usually enough: summarising documents, answering questions about files, comparing contracts, writing drafts.

What matters most is less the model than its connection to your own documents. A local system that searches the organisation's files and backs every answer with its source is often more useful for this work than a bigger model without access to the documents.

Operating and IT effort

Cloud AI is quick to start: create an account, invite users. The vendor takes care of updates and computing power.

A self-built local AI, on the other hand, requires know-how: suitable hardware, setting up the model and document search, permissions, updates. A preconfigured appliance takes most of this effort away. The Sinabox ships set up and is typically productive in about a week.

Which option fits when

• Cloud AI fits when no confidential data is processed, for example for general research or marketing copy.
• Local AI fits when data may not leave the building for legal or contractual reasons: client files, patient data, HR records, trade secrets.
• AI in German data centers fits when you do not want to run your own hardware but processing in Germany is sufficient.

Many organisations combine both: cloud AI for non-critical tasks, local AI for everything confidential. What matters is a clear rule on which data may go into which system.