Local AI or cloud AI: cost, data protection and effort compared

What sets local AI and cloud AI apart

With cloud AI such as ChatGPT, Microsoft Copilot or Google Gemini, the language models run in the vendor's data centers. Every query and every uploaded document is transferred there and processed there.

With local AI, also called on-premises AI, the language model runs on your own hardware, usually an open model on an AI server. Queries and documents stay in your own network. In between sits a third option: AI operated by a German provider in German data centers, without a US cloud.

Cost: per-user licences or a fixed monthly rate

Cloud AI for organisations is usually billed per user and month, so cost grows with every seat. A worked example with an assumed licence of 30 euros per user and month: 20 users cost 600 euros a month, 50 users 1,500 euros.

Local AI mainly costs hardware and setup, and the price does not depend on the number of users. The Sinabox starts at 365 euros a month for the Standard model and at 565 euros for the 19-inch rack, both excluding VAT with a 24-month minimum term. The more employees use the AI, the sooner the fixed rate pays off. How many concurrent users one device supports depends on the model and usage and is sized up front.

Cloud AI licences do not include the cost of data protection assessments, contracts and documentation. For professionals bound by secrecy and for authorities this effort can be considerable.

Data protection and confidentiality

Cloud AI requires a data processing agreement, an assessment of possible third-country transfers and clear rules on whether inputs are stored or used for training. With US vendors the CLOUD Act adds to this, as it can give US authorities access to data even in European data centers. Law firms, tax advisers and doctors in Germany are also bound by Section 203 StGB.

With local AI the data never leaves the building. There is no third-country transfer, no external provider with access to content and no question about training data. Data protection documentation becomes much shorter.

Performance: what local models can do

The largest cloud models are often still ahead of open models on general knowledge, very long texts and creative tasks. For typical office work, however, a local model is usually enough: summarising documents, answering questions about files, comparing contracts, writing drafts.

What matters most is less the model than its connection to your own documents. A local system that searches the organisation's files and backs every answer with its source is often more useful for this work than a bigger model without access to the documents.

Operating and IT effort

Cloud AI is quick to start: create an account, invite users. The vendor takes care of updates and computing power.

A self-built local AI, on the other hand, requires know-how: suitable hardware, setting up the model and document search, permissions, updates. A preconfigured appliance takes most of this effort away. The Sinabox ships set up and is typically productive in about a week.

Which option fits when

• Cloud AI fits when no confidential data is processed, for example for general research or marketing copy.
• Local AI fits when data may not leave the building for legal or contractual reasons: client files, patient data, investigation files, trade secrets.
• AI in German data centers fits when you do not want to run your own hardware but processing in Germany is sufficient.

Many organisations combine both: cloud AI for non-critical tasks, local AI for everything confidential. What matters is a clear rule on which data may go into which system.