Choose a Deployment Shape and Operating Model

Define the workload before choosing a platform

Write the users, tasks, model class, input and output modalities, sensitivity, concurrency, arrival pattern, context size, output length, latency objective, availability target, geographic boundary, disconnected requirement, retention rule, and recovery objective. Separate an experiment from a shared internal service and a production dependency.

Record what happens when the endpoint is slow, unavailable, overloaded, wrong, unsafe, or unable to process a request. A deployment shape is suitable only when the team can operate its normal path and its failure path.

Compare five useful deployment shapes

A workstation can support individual learning, offline experiments, and low-concurrency development, but user sessions, sleep, local privilege, thermal limits, and unmanaged changes make it a weak default for a shared service. A container adds a reproducible process boundary and packaging contract; it does not by itself add strong isolation, authentication, backups, high availability, or resource limits.

On-premises infrastructure can align with local data, connectivity, and hardware-control requirements while transferring capacity, patching, power, cooling, spare-parts, and incident ownership to the operator. Edge deployments reduce distance or support intermittent connectivity but constrain power, memory, update bandwidth, physical security, and repair.

Cloud compute can provide faster provisioning, managed infrastructure options, and elastic capacity, but it still requires verified region, identity, networking, encryption, quotas, cost controls, observability, backup, and exit planning. Hybrid designs add routing and consistency choices. None of these labels proves privacy, cost efficiency, performance, or resilience.

Separate process, host, site, and control-plane failure domains

List the process, container, device, host, accelerator, storage, network, identity provider, scheduler, site, region, registry, and management-plane dependencies. Decide which failures should restart a process, replace a replica, move to another node, fail over to another site, shed load, or stop safely.

Do not claim high availability because multiple replicas exist on one host or because an orchestrator can restart a broken process. Test the actual failure domain and confirm that state, secrets, model artifacts, queues, and routing remain recoverable.

Assign operating ownership before deployment

Name accountable owners for model and license review, artifact integrity, host and accelerator lifecycle, runtime, identity and access, secrets, network policy, data handling, evaluation, capacity, cost, telemetry, vulnerability response, abuse handling, incident command, backup, restore, rollback, and end-of-life.

For each role, record decision authority, on-call expectation, evidence produced, escalation path, and substitute. A technically successful endpoint without owned updates and recovery is an unmanaged liability.

Choose the smallest reversible shape that proves the next claim

Select the initial shape that can safely answer the next decision: whether the exact artifact and runtime work, whether representative quality passes, whether one host meets capacity, or whether a shared service meets reliability. Avoid building multi-site orchestration before a bounded single-endpoint contract is evaluated.

Record the chosen and rejected shapes, assumptions, evidence, unresolved risks, cost and capacity envelope, required controls, stop conditions, migration path, and review date. Trigger re-evaluation when workload, data class, users, geography, model, runtime, demand, service objective, incident history, or ownership changes.