AWS Bedrock Batch Inference: 50% Lower Cost

Bedrock Batch Inference lets you submit large volumes of inference requests as a single job and receive results asynchronously — at 50% off standard on-demand pricing. Instead of paying $0.003/1K input tokens for Claude Sonnet, you pay $0.0015/1K. For document processing pipelines, content generation, and data extraction at scale, this is the single largest cost savings available.

TL;DR: Batch inference processes thousands of requests asynchronously at 50% off on-demand token prices. Input is a JSONL file in S3, output is written to S3. Turnaround time is up to 24 hours (often faster). Use batch for any workload that doesn't need real-time responses: document summarization, content classification, data extraction, email generation, translation pipelines.

---

How Batch Inference Works

1. Prepare: Create JSONL file with all requests → Upload to S3
2. Submit: Create batch inference job via API/console
3. Process: Bedrock processes all requests (up to 24 hours)
4. Retrieve: Results written to S3 as JSONL

Pricing Comparison

Output tokens receive the same 50% discount.

---

Input Format

Each line of the JSONL file is an independent inference request:

{"recordId": "001", "modelInput": {"anthropic_version": "bedrock-2023-05-31", "max_tokens": 500, "messages": [{"role": "user", "content": "Summarize this document: ..."}]}}
{"recordId": "002", "modelInput": {"anthropic_version": "bedrock-2023-05-31", "max_tokens": 500, "messages": [{"role": "user", "content": "Summarize this document: ..."}]}}

Input Limits

---

Output Format

Results are written to S3 as JSONL:

{"recordId": "001", "modelOutput": {"content": [{"text": "The document discusses...", "type": "text"}], "usage": {"input_tokens": 1500, "output_tokens": 200}}}
{"recordId": "002", "modelOutput": {"content": [{"text": "This report covers...", "type": "text"}], "usage": {"input_tokens": 800, "output_tokens": 150}}}

Failed records include error details instead of model output, allowing you to retry only the failures.

---

Real-World Use Cases and Cost Savings

Document Summarization Pipeline

Content Classification

Weekly Report Generation

---

When to Use Batch vs Real-Time

Hybrid Approach

Many production systems use both:

• Real-time: User-facing features (chat, search, recommendations)
• Batch: Background processing (nightly reports, content generation, data enrichment)

---

Best Practices

1. Right-Size Max Tokens

Set max_tokens to the minimum needed for each request. Batch jobs don't benefit from token caching between records, so every excess token is wasted at full output price (even discounted).

2. Use the Cheapest Adequate Model

The 50% discount applies to all models. Claude Haiku at batch pricing ($0.0004/1K input) is extremely cost-effective for classification and extraction tasks.

3. Split Large Jobs

Instead of one 50,000-record job, submit five 10,000-record jobs. This:

• Provides partial results faster
• Reduces blast radius if a job fails
• Allows different models for different batches

4. Implement Record-Level Error Handling

Check every output record for errors. Collect failed record IDs and retry them in a follow-up batch job rather than reprocessing the entire batch.

5. Compress Input Data

Remove unnecessary whitespace, metadata, and formatting from input documents before batching. Every token counts at scale — a 10% reduction in input tokens across 50K documents saves significantly.

---

Related Guides

• AWS Bedrock Cost Optimization Guide
• AWS Bedrock Pricing Guide
• AWS Bedrock Embeddings Guide
• LLM Inference Cost Optimization

---

FAQ

How long do batch jobs take?

Most batch jobs complete within 1-6 hours. The SLA is 24 hours. Smaller jobs (under 1,000 records) often complete within an hour. Processing time scales with total token volume, not record count.

Can I cancel a batch job?

Yes. You can stop a running batch job. Records already processed are available in the output file. Unprocessed records are not billed.

Is batch inference available for all Bedrock models?

Batch is available for most major models including Claude (all versions), Llama, Mistral, and Titan. Check the Bedrock API reference for the current supported model list, as availability expands regularly.

---

Lower Your Bedrock Batch Inference Costs with Wring

Wring helps you access AWS credits and volume discounts to lower your Bedrock batch inference costs. Through group buying power, Wring negotiates better rates so you pay less per batch inference job.

Start saving on Bedrock batch inference →