Name: Granite 4.1 8B
Brand: Ibm Granite
Author: Aivoax

CompanyIbm Granite

Context131K

Released2026-04

Updated2026-09-03

Granite 4.1 8B Overview

Granite 4.1 is IBM’s new open-model family, released on June 17, 2026, in 2B, 8B, and 20B sizes. This review focuses on the 8B model.

IBM offers both base and instruction-tuned versions, with weights released under Apache 2.0. You can download, fine-tune, self-host, and use them commercially without asking IBM for permission.

Granite 4.1 8B has a 128K context window, enough for most enterprise documents, codebases, and long conversations.

Its language support leans Western: English, French, Spanish, German, and Portuguese are the stronger areas. Chinese works, but it is not the model’s strong suit.

IBM is not trying to win every benchmark here. Granite is aimed at the work companies actually need: instruction following, coding, RAG, and tool use.

The results fit that positioning. Granite 4.1 8B ranked fourth on Open LLM Leaderboard v2 and beat similarly sized models such as Llama 4.1 Scout and Qwen 3.5 8B on several internal and external tests.

There is no single headline score that blows everything else away.

For an 8B model, the bigger win is that there are few obvious weak spots.

Granite 4.1 8B Pricing

Plan	Price	Description
IBM watsonx API	Input $0.10 / 1M tokens; Output $0.30 / 1M tokens	IBM-hosted with enterprise SLA and support
IBM watsonx monthly plan	$150/month, including 3M input + 1M output tokens	Better for predictable usage
Together AI	Input $0.10 / 1M tokens; Output $0.30 / 1M tokens	Same listed rate as IBM
Cerebras API	Not publicly listed	Optimized for Cerebras hardware
Self-hosted	Model is free; hardware and ops are not	Apache 2.0

Local deployment is where Granite gets more interesting.

With FP8 quantization, the 8B model needs roughly 8GB of VRAM. The instruction version is around 9GB.

That puts it within reach of consumer GPUs.

Granite 4.1 8B Key Features

1. Instruction Following: Boring in a Good Way

Granite 4.1 8B scores 71.12 on ChatRAG-Bench and stays fairly consistent across multi-turn and long-context tasks.

That kind of reliability matters in business software.

A support bot has to follow format rules. A knowledge assistant has to stay grounded in the source. An internal agent has to call the right tool instead of improvising.

Models that constantly add their own spin can look impressive in demos and become annoying very quickly in production.

Granite is more restrained.

Tell it what to do, and it usually gets on with it.

2. Coding: Solid for 8B

Granite 4.1 8B scores 79.3 on HumanEval and 70.2 on MBPP.

It supports Python, Java, C++, JavaScript, Go, Rust, and other common languages.

IBM also trained it on a large amount of licensed code data.

For companies building internal coding tools, that matters. Compliance can be harder to explain than benchmark scores.

3. RAG and Tool Use: This Is Granite’s Territory

Granite 4.1 8B scores 78.18 on BFCL v3, putting it in a respectable spot among models of similar size.

For an enterprise app, the question is not whether the model sounds natural. It is whether it can query a database, call an internal API, read a knowledge base, and trigger the right business system.

That is where Granite actually matters.

This model makes far more sense as part of a workflow than as a chatbot sitting by itself.

4. 128K Context, With a Language Bias

A 128K context window is not huge by 2026 standards, but it is enough for most contracts, technical documents, code repositories, and long conversations.

Multilingual support is stronger across European languages.

Chinese is weaker.

If Chinese is your main workload, test the model on real internal data before making a deployment decision. English benchmark results will not tell you enough.

Summary

Pros

Apache 2.0 licensing
Roughly 8GB VRAM with FP8
Balanced instruction following, coding, RAG, and tool use
128K context window
Base and instruction-tuned versions
Enterprise SLA and compliance support through IBM

Limitations

Text-only
Chinese support is limited
API pricing is higher than low-cost competitors such as DeepSeek
It does not lead the benchmark charts

Recommended For

Teams that want a stable, deployable enterprise open model
Developers who need local inference for privacy reasons
RAG, knowledge-base, and tool-using applications
Existing IBM watsonx customers
Developers who want an 8B model that can run on consumer hardware

Not Recommended For

Anyone chasing the absolute highest benchmark scores
Chinese-first applications
Very high-volume API workloads where token price is everything
Products that need image, audio, or video support

Granite 4.1 8B