Self-hosted models

Host your own AI infrastructure to use GitLab Duo features with the LLMs of your choice. Use a self-hosted AI Gateway to keep all request and response data in your own environment, avoid external API calls, and manage the full lifecycle of requests to your LLM backends.

Deployment options

You can use self-hosted models with different deployment options.

GitLab Duo (Classic)

GitLab Duo Self-Hosted for customers with GitLab Duo Enterprise who are using GitLab Duo (Classic) features. You can use:

• On-premise models or private cloud-hosted models
• GitLab-managed models in a hybrid deployment

This option uses seat-based pricing.

GitLab Duo Agent Platform

GitLab Duo Agent Platform Self-Hosted to use on-premise models or private cloud-hosted models in the GitLab Duo Agent Platform.

For customers with an offline license, billing is seat based and you must have the GitLab Duo Agent Platform Self-Hosted add-on.

For customers with an online license, billing is usage based. You can also use GitLab-managed models in a hybrid deployment.

Feature versions and status

The following table lists:

• The GitLab version required to use the feature.
• The feature status. A feature status in the deployment might be different to the status listed in the feature.

To use GitLab Duo (Classic) features with GitLab Duo Self-Hosted, you must have the GitLab Duo Enterprise add-on. This applies even if you can use these features with GitLab Duo Core or GitLab Duo Pro when GitLab hosts and connects to those models through the cloud-based AI Gateway.

Feature	GitLab version	Status
GitLab Duo Agent Platform	GitLab 18.8 and later	Generally available
GitLab Duo (Classic)
Code Suggestions (Classic)	GitLab 17.9 and later	Generally available
GitLab Duo Chat (Classic)	GitLab 17.9 and later	Generally available
Code Explanation	GitLab 17.9 and later	Generally available
Test Generation	GitLab 17.9 and later	Generally available
Refactor Code	GitLab 17.9 and later	Generally available
Fix Code	GitLab 17.9 and later	Generally available
Code Review	GitLab 18.3 and later	Generally available
Root Cause Analysis	GitLab 17.10 and later	Beta
Vulnerability Explanation	GitLab 18.1.2 and later	Beta
Merge Commit Message Generation	GitLab 18.1.2 and later	Beta
Merge Request Summary	GitLab 18.1.2 and later	Beta
Discussion Summary	GitLab 18.1.2 and later	Beta
GitLab Duo for the CLI	GitLab 18.1.2 and later	Beta
Vulnerability Resolution	GitLab 18.1.2 and later	Beta
GitLab Duo and SDLC trends Dashboard	GitLab 17.9 and later	Beta
Code Review Summary	GitLab 18.1.2 and later	Experiment

AI Gateway configurations

After you choose a product option, configure how your AI Gateway connects to LLMs:

• Self-hosted AI Gateway and LLMs: Use your own AI Gateway and models for full control over your AI infrastructure.
• Hybrid AI Gateway and model configuration: For each feature, use either your self-hosted AI Gateway with self-hosted models, or the GitLab.com AI Gateway with GitLab AI vendor models.
• GitLab.com AI Gateway with default GitLab external vendor LLMs: Use GitLab managed AI infrastructure.

Configuration	Self-hosted AI Gateway	Hybrid AI Gateway and model configuration	GitLab.com AI Gateway
Infrastructure requirements	Requires hosting your own AI Gateway and models	Requires hosting your own AI Gateway and models	No additional infrastructure needed
Model options	Choose from supported self-hosted models	Choose from supported self-hosted models or GitLab AI vendor models for each GitLab Duo feature	Uses the default GitLab AI vendor models
Network requirements	Can operate in fully isolated networks	Requires internet connectivity for GitLab Duo features that use GitLab AI vendor models	Requires internet connectivity
Responsibilities	You set up your infrastructure, and do your own maintenance	You set up your infrastructure, do your own maintenance, and choose which features use GitLab AI vendor models and AI Gateway	GitLab does the set up and maintenance

Self-hosted AI Gateway and LLMs

In a fully self-hosted configuration, you deploy your own AI Gateway and use only supported LLMs in your infrastructure, without using GitLab infrastructure or AI vendor models. This gives you full control over your data and security.

While you deploy your own AI Gateway, you can still use cloud-based LLM services like AWS Bedrock or Azure OpenAI as your model backend and they will continue to connect through your self-hosted AI Gateway.

If you have an offline environment with physical barriers or security policies that prevent or limit internet access, and comprehensive LLM controls, you should use this fully self-hosted configuration.

For more information, see:

• The self-hosted AI Gateway configuration diagram.

Hybrid AI Gateway and model configuration

In this hybrid configuration, you deploy your own AI Gateway and self-hosted models for most features, but configure specific features to use GitLab AI vendor models. When a feature is configured to use a GitLab AI vendor model, requests for that feature are sent to the GitLab-hosted AI Gateway instead of your self-hosted AI Gateway.

This option provides flexibility by allowing you to:

• Use your own self-hosted models for features where you want full control.
• Use GitLab-managed vendor models for specific features where you prefer the models GitLab has curated.

For more information, see:

• Configure GitLab AI vendor models

GitLab managed models

Use GitLab managed models to connect to AI models without the need to self-host infrastructure. These models are managed entirely by GitLab.

You can select the default GitLab model to use with an AI-native feature. For the default model, GitLab uses the best model based on availability, quality, and reliability. The model used for a feature can change without notice.

When you select a specific GitLab managed model, all requests for that feature use that model exclusively. If the model becomes unavailable, requests to the AI Gateway fail and users cannot use that feature until another model is selected.

GitLab.com AI Gateway with default GitLab external vendor LLMs

If you do not meet the use case criteria for GitLab Duo Self-Hosted, you can use the GitLab.com AI Gateway with default GitLab external vendor LLMs.

The GitLab.com AI Gateway is the default Enterprise offering and is not self-hosted. In this configuration, you connect your instance to the GitLab-hosted AI Gateway, which integrates with external vendor LLM providers, including:

• Anthropic
• Fireworks AI
• Google Vertex

These LLMs communicate through the GitLab Cloud Connector, offering a ready-to-use AI solution without the need for on-premise infrastructure.

For more information, see the GitLab.com AI Gateway configuration diagram.

To set up this infrastructure, see how to configure GitLab Duo on a GitLab Self-Managed instance.

Set up a private infrastructure

If you have an offline license, you can set up a fully private infrastructure:

1. Install a Large Language Model (LLM) serving infrastructure.

   • GitLab supports various platforms for serving and hosting your LLMs, such as vLLM, AWS Bedrock, and Azure OpenAI. For more information about each platform, see supported LLM platforms documentation.

   • GitLab provides a matrix of supported models with their specific features and hardware requirements. For more information, see the supported models and hardware requirements documentation.

2. Install the AI Gateway to access GitLab Duo features.

3. Configure your GitLab instance for features to use self-hosted models.

4. Enable logging to track and manage your system’s performance.

Related topics

• Troubleshooting
• Install the GitLab AI Gateway
• Supported models
• GitLab Duo Self-Hosted supported platforms