Models and hardware requirements

  • Tier: Premium, Ultimate
  • Offering: GitLab Self-Managed, GitLab Dedicated for Government

You can integrate with industry-leading models from Mistral, Meta, Anthropic, and OpenAI through your preferred serving platform.

You can use:

  • Supported models to match your specific performance needs and use cases.
  • In GitLab 18.3 and later, your own compatible model to experiment with models beyond the officially supported options.
  • GitLab-managed models to connect to AI models without the need to host your own infrastructure. These models are managed entirely by GitLab.

Supported models

GitLab-supported models offer different levels of functionality for GitLab Duo features, depending on the specific model and feature combination.

  • check-circle-filled Full functionality: The model can likely handle the feature without any loss of quality.
  • check-circle-dashed Partial functionality: The model supports the feature, but there might be compromises or limitations.
  • dash-circle Limited functionality: The model is unsuitable for the feature, likely resulting in significant quality loss or performance issues. Models that have limited functionality for a feature will not receive GitLab support for that specific feature.

GitLab.com adds new GitLab-managed models as soon as they’re validated on the GitLab-hosted AI Gateway. Self-hosted support for a new model might arrive in a later release after validation with each supported platform (such as AWS Bedrock, Gemini Enterprise Agent Platform, and Azure OpenAI). Any models supported on GitLab.com should also work on GitLab Duo Self-Hosted.

Model series Model name Model family Code completion Code generation GitLab Duo Non-Agentic Chat GitLab Duo Agent Platform
Claude 4 Claude 4 Sonnet Claude check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
Claude 4 Claude Haiku 4.5 Claude check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
Claude 4 Claude Sonnet 4.6 Claude check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
Claude 4 Claude Opus 4.8 Claude check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
Claude 5 Claude Sonnet 5.5 Claude check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
Claude 5 Claude Haiku 5.5 Claude check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
GLM GLM-5.1-FP8 General dash-circle Limited functionality dash-circle Limited functionality check-circle-filled Full functionality check-circle-filled Full functionality
GLM GLM-5.3 General dash-circle Limited functionality dash-circle Limited functionality check-circle-filled Full functionality check-circle-filled Full functionality
GPT GPT-4 Turbo GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-dashed Partial functionality dash-circle Limited functionality
GPT GPT-4o GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality dash-circle Limited functionality
GPT GPT-4o-mini GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-dashed Partial functionality dash-circle Limited functionality
GPT GPT-5 GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
GPT GPT-5 Codex GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
GPT GPT-5 Mini GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-dashed Partial functionality
GPT GPT-5.1 GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
GPT GPT-5.2 GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
GPT GPT-oss-120B GPT check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality dash-circle Limited functionality
Kimi Kimi-K2.5 General dash-circle Limited functionality dash-circle Limited functionality check-circle-dashed Partial functionality check-circle-dashed Partial functionality
Kimi Kimi-K2.6 General dash-circle Limited functionality dash-circle Limited functionality check-circle-dashed Partial functionality check-circle-filled Full functionality
Kimi Kimi-K3 General dash-circle Limited functionality dash-circle Limited functionality check-circle-dashed Partial functionality check-circle-filled Full functionality
Llama Llama 3 8B Llama 3 check-circle-dashed Partial functionality check-circle-filled Full functionality dash-circle Limited functionality dash-circle Limited functionality
Llama Llama 3 70B Llama 3 check-circle-dashed Partial functionality check-circle-filled Full functionality dash-circle Limited functionality dash-circle Limited functionality
Llama Llama 3.1 8B Llama 3 check-circle-dashed Partial functionality check-circle-filled Full functionality check-circle-dashed Partial functionality dash-circle Limited functionality
Llama Llama 3.1 70B Llama 3 check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality dash-circle Limited functionality
Llama Llama 3.3 70B Llama 3 check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality dash-circle Limited functionality
MiniMax MiniMax-M2.7 General dash-circle Limited functionality dash-circle Limited functionality check-circle-dashed Partial functionality check-circle-filled Full functionality
MiniMax MiniMax-M3 General dash-circle Limited functionality dash-circle Limited functionality check-circle-dashed Partial functionality check-circle-filled Full functionality
Mistral Codestral 22B v0.1 Mistral Codestral check-circle-filled Full functionality check-circle-filled Full functionality check-circle-dashed Partial functionality dash-circle Limited functionality
Mistral Mistral Medium 3.5 128B Mistral check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality
Mistral Mistral Small 24B Instruct 2506 Mistral check-circle-filled Full functionality check-circle-filled Full functionality check-circle-filled Full functionality dash-circle Limited functionality
Nemotron Nemotron 3 Ultra 550B-A55B General dash-circle Limited functionality dash-circle Limited functionality check-circle-dashed Partial functionality check-circle-dashed Partial functionality
Qwen Qwen3.8-27B Qwen dash-circle Limited functionality dash-circle Limited functionality check-circle-dashed Partial functionality check-circle-filled Full functionality

Compatible models

  • Status: Beta

You can use your own compatible models and platform with GitLab Duo Agent Platform and GitLab Duo features. These models include ones you host yourself (for example, served through vLLM or LiteLLM), provided that they’re exposed through an OpenAI API-compatible /v1 endpoint. For compatible models and for supported models without their own family (for example, GLM, Kimi, MiniMax, and Nemotron), select the General model family.

Compatible models are excluded from the definition of Customer Integrated Models in the AI Functionality Terms. Compatible models and platforms must adhere to the OpenAI API specification. Models and platforms that have previously been marked as experimental or beta are now considered compatible models.

This feature is in beta and is therefore subject to change as we gather feedback and improve the integration:

  • GitLab does not provide technical support for issues specific to your chosen model or platform.
  • Not all Agent Platform or GitLab Duo features are guaranteed to work optimally with every compatible model.
  • Response quality, speed, and performance overall might vary significantly based on your model choice.

GitLab Duo

Model family Model name
General Any model compatible with the OpenAI API specification
CodeGemma CodeGemma 2b
CodeGemma CodeGemma 7b-it
CodeGemma CodeGemma 7b-code
Code Llama Code-Llama 13b
DeepSeek Coder DeepSeek Coder 33b Instruct
DeepSeek Coder DeepSeek Coder 33b Base

GitLab Duo Agent Platform

Model family Model name
General Any model compatible with the OpenAI API specification
Gemini Gemini 3.1 Pro
Gemini Gemini 3.0 Flash
General Gemma-4-31B-IT
Qwen Qwen3.6-35B-A3B

Context window requirements

With the general model family, GitLab Duo Agent Platform trims conversation history to fit a 200,000-token context window. Other model families set their own context window. Code Review Flow does not trim merge request context to fit that window. It sends up to 1 MiB of merge request context plus its instructions in one prompt. To review large merge requests, serve the model with a larger context window, if the model supports one.

When you serve a model with vLLM, set --max-model-len to match the context window you want to support. vLLM returns an error, and the session fails, if the prompt or the requested output tokens are more than --max-model-len. For sizing guidance, see Example model deployment with vLLM.

Tool calling requirements

GitLab Duo Agent Platform flows and GitLab Duo Agentic Chat send tool definitions to the model. Your model endpoint must support OpenAI-compatible tool calling. GitLab Duo Non-Agentic Chat does not send tools and works without tool calling.

When you serve a model with vLLM, start the server with --enable-auto-tool-choice. Set --tool-call-parser to the parser that matches your model. If you start vLLM without these flags, it rejects each GitLab Duo request that carries tools with an HTTP 400 error:

"auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set

GitLab Duo then shows a generic error, such as Error: ModelError in a flow. To find the cause, check the AI Gateway logs, which include the vLLM error.

A parser that does not match the model can fail without an error. The model returns its tool call as plain text, and the flow finishes without running a tool.

GitLab-managed models

GitLab-managed models integrate with GitLab-hosted AI Gateway infrastructure to provide access to AI models curated and made available by GitLab. Instead of using your own self-hosted models, you can choose to use GitLab-managed models for specific GitLab Duo features.

To choose which features use GitLab-managed models, see select a GitLab-managed model for a feature.

When enabled for a specific feature:

  • All calls to those features configured with a GitLab-managed model use the GitLab-hosted AI Gateway, not the self-hosted AI Gateway.
  • No detailed logs are generated in the GitLab-hosted AI Gateway, even when AI logs are enabled. This prevents unintended leaks of sensitive information.

Hardware requirements

The following hardware specifications are the minimum requirements for running GitLab Duo Self-Hosted. Requirements vary significantly based on the model size and intended usage.

For the hardware requirements for the AI Gateway, see scaling recommendations.

Base system requirements

  • CPU:
    • Minimum: 8 cores (16 threads)
    • Recommended: 16+ cores for production environments
  • RAM:
    • Minimum: 32 GB
    • Recommended: 64 GB for most models
  • Storage:
    • SSD with sufficient space for model weights and data.

GPU requirements by model size

The following requirements are based on NVIDIA A100 GPUs. Newer NVIDIA data center GPUs, such as the H100 and H200, are expected to work but have not been tested.

GPU model Architecture VRAM
A100 Ampere 40/80 GB
H100 Hopper 80/94 GB
H200 Hopper 141 GB

The following table shows minimum configurations with A100 GPUs. An H100 (80 GB) needs the same number of GPUs as an A100 (80 GB). Based on the minimum VRAM required, an H200 (141 GB) reduces the GPU count for larger models:

  • 1x H200 for 22B models
  • 4x H200 for Mixtral 8x22B
Model size Minimum GPU configuration Minimum VRAM required
7B models
(for example, Mistral 7B)
1x NVIDIA A100 (40 GB) 35 GB
22B models
(for example, Codestral 22B)
2x NVIDIA A100 (80 GB) 110 GB
Mixtral 8x7B 2x NVIDIA A100 (80 GB) 220 GB
Mixtral 8x22B 8x NVIDIA A100 (80 GB) 526 GB

Use Hugging Face’s memory utility to verify memory requirements.

Response time by model size and GPU

The following benchmarks cover A100 GPUs only.

Small machine

With a a2-highgpu-2g (2x NVIDIA A100 40 GB - 150 GB vRAM) or equivalent:

Model name Number of requests Average time per request (sec) Average tokens in response Average tokens per second per request Total time for requests Total TPS
Mistral-7B-Instruct-v0.3 1 7.09 717.0 101.19 7.09 101.17
Mistral-7B-Instruct-v0.3 10 8.41 764.2 90.35 13.70 557.80
Mistral-7B-Instruct-v0.3 100 13.97 693.23 49.17 20.81 3331.59

Medium machine

With a a2-ultragpu-4g (4x NVIDIA A100 40 GB - 340 GB vRAM) machine on GCP or equivalent:

Model name Number of requests Average time per request (sec) Average tokens in response Average tokens per second per request Total time for requests Total TPS
Mistral-7B-Instruct-v0.3 1 3.80 499.0 131.25 3.80 131.23
Mistral-7B-Instruct-v0.3 10 6.00 740.6 122.85 8.19 904.22
Mistral-7B-Instruct-v0.3 100 11.71 695.71 59.06 15.54 4477.34
Mixtral-8x7B-Instruct-v0.1 1 6.50 400.0 61.55 6.50 61.53
Mixtral-8x7B-Instruct-v0.1 10 16.58 768.9 40.33 32.56 236.13
Mixtral-8x7B-Instruct-v0.1 100 25.90 767.38 26.87 55.57 1380.68

Large machine

With a a2-ultragpu-8g (8 x NVIDIA A100 80 GB - 1360 GB vRAM) machine on GCP or equivalent:

Model name Number of requests Average time per request (sec) Average tokens in response Average tokens per second per request Total time for requests (sec) Total TPS
Mistral-7B-Instruct-v0.3 1 3.23 479.0 148.41 3.22 148.36
Mistral-7B-Instruct-v0.3 10 4.95 678.3 135.98 6.85 989.11
Mistral-7B-Instruct-v0.3 100 10.14 713.27 69.63 13.96 5108.75
Mixtral-8x7B-Instruct-v0.1 1 6.08 709.0 116.69 6.07 116.64
Mixtral-8x7B-Instruct-v0.1 10 9.95 645.0 63.68 13.40 481.06
Mixtral-8x7B-Instruct-v0.1 100 13.83 585.01 41.80 20.38 2869.12
Mixtral-8x22B-Instruct-v0.1 1 14.39 828.0 57.56 14.38 57.55
Mixtral-8x22B-Instruct-v0.1 10 20.57 629.7 30.24 28.02 224.71
Mixtral-8x22B-Instruct-v0.1 100 27.58 592.49 21.34 36.80 1609.85