Use open weight models as your AI coding agent with Amazon Bedrock

來源原標題

Use open weight models as your AI coding agent with Amazon Bedrock

作者

Aris Tsakpinis

原始標籤

Advanced (300) · Amazon Bedrock · Technical How-to

原始連結

https://aws.amazon.com/blogs/machine-learning/use-open-weight-models-as-your-ai-coding-agent-with-amazon-bedrock/

摘要原文

AI coding agents have become a core part of how developers write, debug, and refactor software. Open weight models on Amazon Bedrock now make these agents practical to run privately and cost-effectively. But most options require you to send your proprietary data to a third-party API, lock you into a single model provider, or charge per-seat subscriptions regardless of how much you use them. If you have data residency requirements, cost-sensitive workloads, or a need for model flexibility, these constraints create real friction. What if you could run an AI coding agent that keeps your data in your own AWS account, switches between frontier open weight models on demand, and charges only for what you consume? OpenCode is an open source, terminal-native AI coding agent built in Go. It reads and edits files, runs shell commands, and understands project structure through Language Server Protocol (LSP) diagnostics. It connects to over 75 large language model (LLM) providers including Amazon Bedrock . When you pair OpenCode with open weight models on Bedrock, you get a coding assistant that runs locally while inference happens securely within your AWS account. There’s no infrastructure to manage and no per-seat fees. In this post, we show you how to set up OpenCode with open weight models on Amazon Bedrock, configure multi-model workflows that match the right model to each task, and walk through practical coding examples using Moonshot AI Kimi K3 , OpenAI GPT-OSS 120B , and NVIDIA Nemotron 3 Super 120B . We also share how Ethara.AI deploys this architecture in production with multi-agent orchestration to power AI engineering and research workflows at scale. The industry is shifting toward open weight models. According to McKinsey’s Open-source technology in the age of AI report (2025) , 76 percent of organizations expect to increase open source AI usage, and leading AI adopters are 40 percent more likely to use open weight models. For coding workloads, five factors drive this shift: Performance parity: Fine-tuned open weight models can outperform proprietary alternatives on domain-specific tasks. CrowdStrike’s fine-tuned NVIDIA Nemotron achieved 96% valid query accuracy, outperforming GPT-4o (61%) and Claude Sonnet 4.5 (94%). Cost efficiency: According to Gartner’s 2026 analysis , agentic workflows multiply token consumption 5–30x, making cost-per-token critical. At scale, on the order of multimillion conversations per month, switching to open weight models on Bedrock can reduce annualized costs. Customization and control: Open weights support fine-tuning, distillation, and domain adaptation. Smaller models can replace expensive general-purpose ones while maintaining quality. Model flexibility: With open weights, you can adopt the right model for each task and evolve as new ones emerge. Switching models is a single API parameter change on Amazon Bedrock . Transparency: Inspectable model architecture and behavior supports regulated industries with AI governance requirements. Amazon Bedrock provides fully managed, serverless access to open weight models. There’s no GPU provisioning or inference infrastructure to manage. For enterprise coding workflows, Bedrock offers several advantages over self-hosting or direct model providers: Data residency and compliance: Code, prompts, and responses stay in your AWS account. Models accessed through an in-Region or geographic profile run in that Region or geography. You can invoke Kimi K3 through a cross-Region inference profile. For workloads without regional restrictions, we recommend using the global profile, global.moonshotai.kimi-k3 , which routes each request to any supported commercial AWS Region worldwide. Global cross-Region inference costs approximately 10% less than a geographic profile. The US geographic profile, us.moonshotai.kimi-k3 , keeps processing within the US geography for data residency requirements. Amazon Bedrock is in scope for common compliance programs including HIPAA, SOC 2, ISO 27001, FedRAMP, and GDPR. For the full list, see AWS services in scope by compliance program . Enterprise security controls: Open weight models inherit the same AWS Identity and Access Management (IAM) policies, AWS CloudTrail logging, AWS PrivateLink connectivity, and encryption controls as proprietary models. No separate security stack required. Flexible pricing: Three tiers match cost to workload: Priority for latency-sensitive production, Standard for on-demand inference (pay per token), and Flex at 50 percent lower cost for variable-latency workloads. No model training on your data: Bedrock doesn’t use your inputs or outputs to train or improve foundation models (FMs). High default capacity: Default limits of 100M tokens per minute and 10K requests per minute help reduce throughput bottlenecks as teams scale. Not every coding task needs the same model. One of the key advantages of using OpenCode with Bedrock is the ability to select and switch between models based on what you’re doing. Where to evaluate models: The Artificial Analysis Coding Index provides a composite benchmark across real-world software engineering tasks (SWE-Bench, Terminal-Bench, SWE-Atlas). Use it to compare model performance, cost per task, and latency. For evaluations against your own prompts and data, you can use Amazon Bedrock Evaluations to run side-by-side comparisons with automatic scoring, LLM-as-a-judge , or human review. For this post, we feature three models that cover the spectrum: The architecture has two parts: OpenCode runs as a terminal user interface (TUI) on your local machine and calls the Amazon Bedrock Converse API for inference. Bedrock hosts the models as fully managed, serverless endpoints. Figure 1: Solution architecture for OpenCode with open weight models on Amazon Bedrock. The developer interacts with OpenCode in the terminal, which sends requests through the Bedrock Converse API to open weight models. AWS IAM authenticates each request, and AWS CloudTrail logs API activity OpenCode’s agent architecture supports assigning different models to different roles: a reasoning model for planning and a faster model for code generation. This creates a multi-model workflow within a single session. Before you start, make sure you have: This section walks through installing the OpenCode CLI and configuring it to authenticate with Amazon Bedrock. Choose one of the following methods: OpenCode’s Bedrock provider uses the standard AWS credential chain. Configure one of the following options: Replace these with your own credentials. For production use, prefer IAM Identity Center or IAM roles over long-lived access keys. You can also store these in a .env file in your project root for persistent configuration. With OpenCode’s agent system, you can assign different models to different roles. Create or edit opencode.json in your project root: This configuration routes planning and architecture tasks (which benefit from deep reasoning) to Kimi K3, while code generation and implementation go to Nemotron 3 Super 120B for optimized throughput. The top-level model field sets GPT-OSS 120B as the default for other context. To browse available Bedrock models interactively, launch OpenCode and enter /models . The following examples show how to match each model to the kind of task it handles best. GPT-OSS 120B is OpenAI’s 120-billion parameter open weight model. It combines strong reasoning with code generation, making it well-suited for architecturally complex implementations that span multiple files. With the multi-model configuration from the previous section, you can override the default model inline or use /model to switch explicitly: Figure 2: OpenCode generating the event-sourced CQRS order service with GPT-OSS 120B OpenCode routes this to GPT-OSS 120B through the Bedrock Converse API with IAM authentication.