niuniu/lijiaoqiao
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: initial public snapshot for github upload

Browse Source
This commit is contained in:
Your Name
2026-03-26 20:06:14 +08:00
commit 0e5ecd930e
3497 changed files with 1586236 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

317
llm-gateway-competitors/litellm-wheel-src/litellm/integrations/gitlab/README.md Normal file
View File

@@ -0,0 +1,317 @@
# LiteLLM gitlab Prompt Management
A powerful prompt management system for LiteLLM that fetches `.prompt` files from gitlab repositories. This enables team-based prompt management with gitlab's built-in access control and version control capabilities.
## Features
- **🏢 Team-based access control**: Leverage gitlab's workspace and repository permissions
- **📁 Repository-based prompt storage**: Store prompts in gitlab repositories
- **🔐 Multiple authentication methods**: Support for access tokens and basic auth
- **🎯 YAML frontmatter**: Define model, parameters, and schemas in file headers
- **🔧 Handlebars templating**: Use `{{variable}}` syntax with Jinja2 backend
- **✅ Input validation**: Automatic validation against defined schemas
- **🔗 LiteLLM integration**: Works seamlessly with `litellm.completion()`
- **💬 Smart message parsing**: Converts prompts to proper chat messages
- **⚙️ Parameter extraction**: Automatically applies model settings from prompts
## Quick Start
### 1. Set up gitlab Repository
Create a repository in your gitlab workspace and add `.prompt` files:
```
your-repo/
├── prompts/
│ ├── chat_assistant.prompt
│ ├── code_reviewer.prompt
│ └── data_analyst.prompt
```
### 2. Create a `.prompt` file
Create a file called `prompts/chat_assistant.prompt`:
```yaml
---
model: gpt-4
temperature: 0.7
max_tokens: 150
input:
schema:
user_message: string
system_context?: string
---
{% if system_context %}System: {{system_context}}
{% endif %}User: {{user_message}}
```
### 3. Configure gitlab Access
#### Option A: Access Token (Recommended)
```python
import litellm
# Configure gitlab access
gitlab_config = {
"project": "a/b/<repo_name>",
"access_token": "your-access-token",
"base_url": "gitlab url",
"prompts_path": "src/prompts", # folder to point to, defaults to root
"branch":"main" # optional, defaults to main
}
# Set global gitlab configuration
litellm.set_global_gitlab_config(gitlab_config)
```
#### Option B: Basic Authentication
```python
import litellm
# Configure gitlab access with basic auth
gitlab_config = {
"project": "a/b/<repo_name>",
"base_url": "base url",
"access_token": "your-app-password", # Use app password for basic auth
"branch": "main",
"prompts_path": "src/prompts", # folder to point to, defaults to root
}
litellm.set_global_gitlab_config(gitlab_config)
```
### 4. Use with LiteLLM
```python
# Use with completion - the model prefix 'gitlab/' tells LiteLLM to use gitlab prompt management
response = litellm.completion(
model="gitlab/gpt-4", # The actual model comes from the .prompt file
prompt_id="prompts/chat_assistant", # Location of the prompt file
prompt_variables={
"user_message": "What is machine learning?",
"system_context": "You are a helpful AI tutor."
},
# Any additional messages will be appended after the prompt
messages=[{"role": "user", "content": "Please explain it simply."}]
)
print(response.choices[0].message.content)
```
## Proxy Server Configuration
### 1. Create a `.prompt` file
Create `prompts/hello.prompt`:
```yaml
---
model: gpt-4
temperature: 0.7
---
System: You are a helpful assistant.
User: {{user_message}}
```
### 2. Setup config.yaml
```yaml
model_list:
- model_name: my-gitlab-model
litellm_params:
model: gitlab/gpt-4
prompt_id: "prompts/hello"
api_key: os.environ/OPENAI_API_KEY
litellm_settings:
global_gitlab_config:
workspace: "your-workspace"
repository: "your-repo"
access_token: "your-access-token"
branch: "main"
```
### 3. Start the proxy
```bash
litellm --config config.yaml --detailed_debug
```
### 4. Test it!
```bash
curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
"model": "my-gitlab-model",
"messages": [{"role": "user", "content": "IGNORED"}],
"prompt_variables": {
"user_message": "What is the capital of France?"
}
}'
```
## Prompt File Format
### Basic Structure
```yaml
---
# Model configuration
model: gpt-4
temperature: 0.7
max_tokens: 500
# Input schema (optional)
input:
schema:
user_message: string
system_context?: string
---
System: You are a helpful {{role}} assistant.
User: {{user_message}}
```
### Advanced Features
**Multi-role conversations:**
```yaml
---
model: gpt-4
temperature: 0.3
---
System: You are a helpful coding assistant.
User: {{user_question}}
```
**Dynamic model selection:**
```yaml
---
model: "{{preferred_model}}" # Model can be a variable
temperature: 0.7
---
System: You are a helpful assistant specialized in {{domain}}.
User: {{user_message}}
```
## Team-Based Access Control
gitlab's built-in permission system provides team-based access control:
1. **Workspace-level permissions**: Control access to entire workspaces
2. **Repository-level permissions**: Control access to specific repositories
3. **Branch-level permissions**: Control access to specific branches
4. **User and group management**: Manage team members and their access levels
### Setting up Team Access
1. **Create workspaces for each team**:
```
team-a-prompts/
team-b-prompts/
team-c-prompts/
```
2. **Configure repository permissions**:
- Grant read access to team members
- Grant write access to prompt maintainers
- Use branch protection rules for production prompts
3. **Use different access tokens**:
- Each team can have their own access token
- Tokens can be scoped to specific repositories
- Use app passwords for additional security
## API Reference
### gitlab Configuration
```python
gitlab_config = {
"workspace": str, # Required: gitlab workspace name
"repository": str, # Required: Repository name
"access_token": str, # Required: gitlab access token or app password
"branch": str, # Optional: Branch to fetch from (default: "main")
"base_url": str, # Optional: Custom gitlab API URL
"auth_method": str, # Optional: "token" or "basic" (default: "token")
"username": str, # Optional: Username for basic auth
"base_url" : str # Optional: Incase where the base url is not https://api.gitlab.org/2.0
}
```
### LiteLLM Integration
```python
response = litellm.completion(
model="gitlab/<base_model>", # required (e.g., gitlab/gpt-4)
prompt_id=str, # required - the .prompt filename without extension
prompt_variables=dict, # optional - variables for template rendering
gitlab_config=dict, # optional - gitlab configuration (if not set globally)
messages=list, # optional - additional messages
)
```
## Error Handling
The gitlab integration provides detailed error messages for common issues:
- **Authentication errors**: Invalid access tokens or credentials
- **Permission errors**: Insufficient access to workspace/repository
- **File not found**: Missing .prompt files
- **Network errors**: Connection issues with gitlab API
## Security Considerations
1. **Access Token Security**: Store access tokens securely using environment variables or secret management systems
2. **Repository Permissions**: Use gitlab's permission system to control access
3. **Branch Protection**: Protect main branches from unauthorized changes
4. **Audit Logging**: gitlab provides audit logs for all repository access
## Troubleshooting
### Common Issues
1. **"Access denied" errors**: Check your gitlab permissions for the workspace and repository
2. **"Authentication failed" errors**: Verify your access token or credentials
3. **"File not found" errors**: Ensure the .prompt file exists in the specified branch
4. **Template rendering errors**: Check your Handlebars syntax in the .prompt file
### Debug Mode
Enable debug logging to troubleshoot issues:
```python
import litellm
litellm.set_verbose = True
# Your gitlab prompt calls will now show detailed logs
response = litellm.completion(
model="gitlab/gpt-4",
prompt_id="your_prompt",
prompt_variables={"key": "value"}
)
```
## Migration from File-Based Prompts
If you're currently using file-based prompts with the dotprompt integration, you can easily migrate to gitlab:
1. **Upload your .prompt files** to a gitlab repository
2. **Update your configuration** to use gitlab instead of local files
3. **Set up team access** using gitlab's permission system
4. **Update your code** to use `gitlab/` model prefix instead of `dotprompt/`
This provides better collaboration, version control, and team-based access control for your prompts.

94
llm-gateway-competitors/litellm-wheel-src/litellm/integrations/gitlab/__init__.py Normal file
View File

@@ -0,0 +1,94 @@
from typing import TYPE_CHECKING, Optional, Dict, Any
if TYPE_CHECKING:
from .gitlab_prompt_manager import GitLabPromptManager
from litellm.types.prompts.init_prompts import PromptLiteLLMParams, PromptSpec
from litellm.integrations.custom_prompt_management import CustomPromptManagement
from litellm.types.prompts.init_prompts import SupportedPromptIntegrations
from litellm.integrations.custom_prompt_management import CustomPromptManagement
from litellm.types.prompts.init_prompts import PromptSpec, PromptLiteLLMParams
from .gitlab_prompt_manager import GitLabPromptManager, GitLabPromptCache
# Global instances
global_gitlab_config: Optional[dict] = None
def set_global_gitlab_config(config: dict) -> None:
"""
Set the global gitlab configuration for prompt management.
Args:
config: Dictionary containing gitlab configuration
- workspace: gitlab workspace name
- repository: Repository name
- access_token: gitlab access token
- branch: Branch to fetch prompts from (default: main)
"""
import litellm
litellm.global_gitlab_config = config # type: ignore
def prompt_initializer(
litellm_params: "PromptLiteLLMParams", prompt_spec: "PromptSpec"
) -> "CustomPromptManagement":
"""
Initialize a prompt from a Gitlab repository.
"""
gitlab_config = getattr(litellm_params, "gitlab_config", None)
prompt_id = getattr(litellm_params, "prompt_id", None)
if not gitlab_config:
raise ValueError("gitlab_config is required for gitlab prompt integration")
try:
gitlab_prompt_manager = GitLabPromptManager(
gitlab_config=gitlab_config,
prompt_id=prompt_id,
)
return gitlab_prompt_manager
except Exception as e:
raise e
def _gitlab_prompt_initializer(
litellm_params: PromptLiteLLMParams,
prompt: PromptSpec,
) -> CustomPromptManagement:
"""
Build a GitLab-backed prompt manager for this prompt.
Expected fields on litellm_params:
- prompt_integration="gitlab" (handled by the caller)
- gitlab_config: Dict[str, Any] (project/access_token/branch/prompts_path/etc.)
- git_ref (optional): per-prompt tag/branch/SHA override
"""
# You can store arbitrary integration-specific config on PromptLiteLLMParams.
# If your dataclass doesn't have these attributes, add them or put inside
# `litellm_params.extra` and pull them from there.
gitlab_config: Dict[str, Any] = getattr(litellm_params, "gitlab_config", None) or {}
git_ref: Optional[str] = getattr(litellm_params, "git_ref", None)
if not gitlab_config:
raise ValueError("gitlab_config is required for gitlab prompt integration")
# prompt.prompt_id can map to a file path under prompts_path (e.g. "chat/greet/hi")
return GitLabPromptManager(
gitlab_config=gitlab_config,
prompt_id=prompt.prompt_id,
ref=git_ref,
)
prompt_initializer_registry = {
SupportedPromptIntegrations.GITLAB.value: _gitlab_prompt_initializer,
}
# Export public API
__all__ = [
"GitLabPromptManager",
"GitLabPromptCache",
"set_global_gitlab_config",
"global_gitlab_config",
]

309
llm-gateway-competitors/litellm-wheel-src/litellm/integrations/gitlab/gitlab_client.py Normal file
View File

@@ -0,0 +1,309 @@
"""
GitLab API client for fetching files from GitLab repositories.
Now supports selecting a tag via `config["tag"]`; falls back to branch ("main").
"""
import base64
from typing import Any, Dict, List, Optional
from urllib.parse import quote
from litellm.llms.custom_httpx.http_handler import HTTPHandler
class GitLabClient:
"""
Client for interacting with the GitLab API to fetch files.
Supports:
- Authentication with personal/access tokens or OAuth bearer tokens
- Fetching file contents from repositories (raw endpoint with JSON fallback)
- Namespace/project path or numeric project ID addressing
- Ref selection via tag (preferred) or branch (default "main")
- Directory listing via the repository tree API
"""
def __init__(self, config: Dict[str, Any]):
"""
Initialize the GitLab client.
Args:
config: Dictionary containing:
- project: Project path ("group/subgroup/repo") or numeric project ID (str|int) [required]
- access_token: GitLab personal/access token or OAuth token [required] (str)
- auth_method: 'token' (default; sends Private-Token) or 'oauth' (Authorization: Bearer)
- tag: Tag name to fetch from (takes precedence over branch if provided)
- branch: Branch to fetch from (default: "main")
- base_url: Base GitLab API URL (default: "https://gitlab.com/api/v4")
"""
project = config.get("project")
access_token = config.get("access_token")
if project is None or access_token is None:
raise ValueError("project and access_token are required")
self.project: str | int = project
self.access_token: str = str(access_token)
self.auth_method = config.get("auth_method", "token") # 'token' or 'oauth'
self.branch = config.get("branch", None)
if not self.branch:
self.branch = "main"
self.tag = config.get("tag")
self.base_url = config.get("base_url", "https://gitlab.com/api/v4")
if not all([self.project, self.access_token]):
raise ValueError("project and access_token are required")
# Effective ref: prefer tag if provided, else branch ("main")
self.ref = str(self.tag or self.branch)
# Build headers
self.headers = {
"Accept": "application/json",
"Content-Type": "application/json",
}
if self.auth_method == "oauth":
self.headers["Authorization"] = f"Bearer {self.access_token}"
else:
# Default GitLab token header
self.headers["Private-Token"] = self.access_token
# Project identifier must be URL-encoded (slashes become %2F)
self._project_enc = quote(str(self.project), safe="")
# HTTP handler
self.http_handler = HTTPHandler()
# ------------------------
# Core helpers
# ------------------------
def _file_raw_url(self, file_path: str, *, ref: Optional[str] = None) -> str:
file_enc = quote(file_path, safe="")
ref_q = quote(ref or self.ref, safe="")
return f"{self.base_url}/projects/{self._project_enc}/repository/files/{file_enc}/raw?ref={ref_q}"
def _file_json_url(self, file_path: str, *, ref: Optional[str] = None) -> str:
file_enc = quote(file_path, safe="")
ref_q = quote(ref or self.ref, safe="")
return f"{self.base_url}/projects/{self._project_enc}/repository/files/{file_enc}?ref={ref_q}"
def _tree_url(
self,
directory_path: str = "",
recursive: bool = False,
*,
ref: Optional[str] = None,
) -> str:
path_q = f"&path={quote(directory_path, safe='')}" if directory_path else ""
rec_q = "&recursive=true" if recursive else ""
ref_q = quote(ref or self.ref, safe="")
return f"{self.base_url}/projects/{self._project_enc}/repository/tree?ref={ref_q}{path_q}{rec_q}"
# ------------------------
# Public API
# ------------------------
def set_ref(self, ref: str) -> None:
"""Override the default ref (tag/branch) for subsequent calls."""
if not ref:
raise ValueError("ref must be a non-empty string")
self.ref = ref
def get_file_content(
self, file_path: str, *, ref: Optional[str] = None
) -> Optional[str]:
"""
Fetch the content of a file from the GitLab repository at the given ref
(tag, branch, or commit SHA). If `ref` is None, uses self.ref.
Strategy:
1) Try the RAW endpoint (returns bytes of the file)
2) Fallback to the JSON endpoint (returns base64-encoded content)
Returns:
File content as UTF-8 string, or None if file not found.
"""
raw_url = self._file_raw_url(file_path, ref=ref)
try:
resp = self.http_handler.get(raw_url, headers=self.headers)
if resp.status_code == 404:
# Fallback to JSON endpoint
return self._get_file_content_via_json(file_path, ref=ref)
resp.raise_for_status()
ctype = (resp.headers.get("content-type") or "").lower()
if (
ctype.startswith("text/")
or "charset=" in ctype
or ctype.startswith("application/json")
):
return resp.text
try:
return resp.content.decode("utf-8")
except Exception:
return resp.content.decode("utf-8", errors="replace")
except Exception as e:
status = getattr(getattr(e, "response", None), "status_code", None)
if status == 404:
return None
if status == 403:
raise Exception(
f"Access denied to file '{file_path}'. Check your GitLab permissions for project '{self.project}'."
)
if status == 401:
raise Exception(
"Authentication failed. Check your GitLab token and auth_method."
)
raise Exception(f"Failed to fetch file '{file_path}': {e}")
def _get_file_content_via_json(
self, file_path: str, *, ref: Optional[str] = None
) -> Optional[str]:
"""
Fallback for get_file_content(): use the JSON file API which returns base64 content.
"""
json_url = self._file_json_url(file_path, ref=ref)
try:
resp = self.http_handler.get(json_url, headers=self.headers)
if resp.status_code == 404:
return None
resp.raise_for_status()
data = resp.json()
content = data.get("content")
encoding = data.get("encoding", "")
if content and encoding == "base64":
try:
return base64.b64decode(content).decode("utf-8")
except Exception:
return base64.b64decode(content).decode("utf-8", errors="replace")
return content
except Exception as e:
status = getattr(getattr(e, "response", None), "status_code", None)
if status == 404:
return None
if status == 403:
raise Exception(
f"Access denied to file '{file_path}'. Check your GitLab permissions for project '{self.project}'."
)
if status == 401:
raise Exception(
"Authentication failed. Check your GitLab token and auth_method."
)
raise Exception(
f"Failed to fetch file '{file_path}' via JSON endpoint: {e}"
)
def list_files(
self,
directory_path: str = "",
file_extension: str = ".prompt",
recursive: bool = False,
*,
ref: Optional[str] = None,
) -> List[str]:
"""
List files in a directory with a specific extension using the repository tree API.
Args:
directory_path: Directory path in the repository (empty for repo root)
file_extension: File extension to filter by (default: .prompt)
recursive: If True, traverses subdirectories
ref: Optional override (tag/branch/SHA). Defaults to self.ref.
Returns:
List of file paths (relative to repo root)
"""
url = self._tree_url(directory_path, recursive=recursive, ref=ref)
try:
resp = self.http_handler.get(url, headers=self.headers)
if resp.status_code == 404:
return []
resp.raise_for_status()
data = resp.json() or []
files: List[str] = []
for item in data:
if item.get("type") == "blob":
file_path = item.get("path", "")
if not file_extension or file_path.endswith(file_extension):
files.append(file_path)
return files
except Exception as e:
status = getattr(getattr(e, "response", None), "status_code", None)
if status == 404:
return []
if status == 403:
raise Exception(
f"Access denied to directory '{directory_path}'. Check your GitLab permissions for project '{self.project}'."
)
if status == 401:
raise Exception(
"Authentication failed. Check your GitLab token and auth_method."
)
raise Exception(f"Failed to list files in '{directory_path}': {e}")
def get_repository_info(self) -> Dict[str, Any]:
"""Get information about the project/repository."""
url = f"{self.base_url}/projects/{self._project_enc}"
try:
resp = self.http_handler.get(url, headers=self.headers)
resp.raise_for_status()
return resp.json()
except Exception as e:
raise Exception(f"Failed to get repository info: {e}")
def test_connection(self) -> bool:
"""Test the connection to the GitLab project."""
try:
self.get_repository_info()
return True
except Exception:
return False
def get_branches(self) -> List[Dict[str, Any]]:
"""Get list of branches in the repository."""
url = f"{self.base_url}/projects/{self._project_enc}/repository/branches"
try:
resp = self.http_handler.get(url, headers=self.headers)
resp.raise_for_status()
data = resp.json()
return data if isinstance(data, list) else []
except Exception as e:
raise Exception(f"Failed to get branches: {e}")
def get_file_metadata(
self, file_path: str, *, ref: Optional[str] = None
) -> Optional[Dict[str, Any]]:
"""
Get minimal metadata about a file via RAW endpoint headers at a given ref.
Args:
file_path: Path to the file in the repository.
ref: Optional override (tag/branch/SHA). Defaults to self.ref.
"""
url = self._file_raw_url(file_path, ref=ref)
try:
headers = dict(self.headers)
headers["Range"] = "bytes=0-0"
resp = self.http_handler.get(url, headers=headers)
if resp.status_code == 404:
return None
resp.raise_for_status()
return {
"content_type": resp.headers.get("content-type"),
"content_length": resp.headers.get("content-length"),
"last_modified": resp.headers.get("last-modified"),
}
except Exception as e:
status = getattr(getattr(e, "response", None), "status_code", None)
if status == 404:
return None
raise Exception(f"Failed to get file metadata for '{file_path}': {e}")
def close(self):
"""Close the HTTP handler to free resources."""
if hasattr(self, "http_handler"):
self.http_handler.close()

760
llm-gateway-competitors/litellm-wheel-src/litellm/integrations/gitlab/gitlab_prompt_manager.py Normal file
View File

@@ -0,0 +1,760 @@
"""
GitLab prompt manager with configurable prompts folder.
"""
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple, Union
from jinja2 import DictLoader, Environment, select_autoescape
from litellm.integrations.custom_prompt_management import CustomPromptManagement
if TYPE_CHECKING:
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
else:
LiteLLMLoggingObj = Any
from litellm.integrations.gitlab.gitlab_client import GitLabClient
from litellm.integrations.prompt_management_base import (
PromptManagementBase,
PromptManagementClient,
)
from litellm.types.llms.openai import AllMessageValues
from litellm.types.prompts.init_prompts import PromptSpec
from litellm.types.utils import StandardCallbackDynamicParams
GITLAB_PREFIX = "gitlab::"
def encode_prompt_id(raw_id: str) -> str:
"""Convert GitLab path IDs like 'invoice/extract' → 'gitlab::invoice::extract'"""
if raw_id.startswith(GITLAB_PREFIX):
return raw_id # already encoded
return f"{GITLAB_PREFIX}{raw_id.replace('/', '::')}"
def decode_prompt_id(encoded_id: str) -> str:
"""Convert 'gitlab::invoice::extract' → 'invoice/extract'"""
if not encoded_id.startswith(GITLAB_PREFIX):
return encoded_id
return encoded_id[len(GITLAB_PREFIX) :].replace("::", "/")
class GitLabPromptTemplate:
def __init__(
self,
template_id: str,
content: str,
metadata: Dict[str, Any],
model: Optional[str] = None,
):
self.template_id = template_id
self.content = content
self.metadata = metadata
self.model = model or metadata.get("model")
self.temperature = metadata.get("temperature")
self.max_tokens = metadata.get("max_tokens")
self.input_schema = metadata.get("input", {}).get("schema", {})
self.optional_params = {
k: v for k, v in metadata.items() if k not in ["model", "input", "content"]
}
def __repr__(self):
return f"GitLabPromptTemplate(id='{self.template_id}', model='{self.model}')"
class GitLabTemplateManager:
"""
Manager for loading and rendering .prompt files from GitLab repositories.
New: supports `prompts_path` (or `folder`) in gitlab_config to scope where prompts live.
"""
def __init__(
self,
gitlab_config: Dict[str, Any],
prompt_id: Optional[str] = None,
ref: Optional[str] = None,
gitlab_client: Optional[GitLabClient] = None,
):
self.gitlab_config = dict(gitlab_config)
self.prompt_id = prompt_id
self.prompts: Dict[str, GitLabPromptTemplate] = {}
self.gitlab_client = gitlab_client or GitLabClient(self.gitlab_config)
if ref:
self.gitlab_client.set_ref(ref)
# Folder inside repo to look for prompts (e.g., "prompts" or "prompts/chat")
self.prompts_path: str = (
self.gitlab_config.get("prompts_path")
or self.gitlab_config.get("folder")
or ""
).strip("/")
self.jinja_env = Environment(
loader=DictLoader({}),
autoescape=select_autoescape(["html", "xml"]),
variable_start_string="{{",
variable_end_string="}}",
block_start_string="{%",
block_end_string="%}",
comment_start_string="{#",
comment_end_string="#}",
)
if self.prompt_id:
self._load_prompt_from_gitlab(self.prompt_id)
# ---------- path helpers ----------
def _id_to_repo_path(self, prompt_id: str) -> str:
"""Map a prompt_id to a repo path (respects prompts_path and adds .prompt)."""
prompt_id = decode_prompt_id(prompt_id)
if self.prompts_path:
return f"{self.prompts_path}/{prompt_id}.prompt"
return f"{prompt_id}.prompt"
def _repo_path_to_id(self, repo_path: str) -> str:
"""
Map a repo path like 'prompts/chat/greeting.prompt' to an ID relative
to prompts_path without the extension (e.g., 'chat/greeting').
"""
path = repo_path.strip("/")
if self.prompts_path and path.startswith(self.prompts_path.strip("/") + "/"):
path = path[len(self.prompts_path.strip("/")) + 1 :]
if path.endswith(".prompt"):
path = path[: -len(".prompt")]
return encode_prompt_id(path)
# ---------- loading ----------
def _load_prompt_from_gitlab(
self, prompt_id: str, *, ref: Optional[str] = None
) -> None:
"""Load a specific .prompt file from GitLab (scoped under prompts_path if set)."""
try:
# prompt_id = decode_prompt_id(prompt_id)
file_path = self._id_to_repo_path(prompt_id)
prompt_content = self.gitlab_client.get_file_content(file_path, ref=ref)
if prompt_content:
template = self._parse_prompt_file(prompt_content, prompt_id)
self.prompts[prompt_id] = template
except Exception as e:
raise Exception(
f"Failed to load prompt '{encode_prompt_id(prompt_id)}' from GitLab: {e}"
)
def load_all_prompts(self, *, recursive: bool = True) -> List[str]:
"""
Eagerly load all .prompt files from prompts_path. Returns loaded IDs.
"""
files = self.list_templates(recursive=recursive)
loaded: List[str] = []
for pid in files:
if pid not in self.prompts:
self._load_prompt_from_gitlab(pid)
loaded.append(pid)
return loaded
# ---------- parsing & rendering ----------
def _parse_prompt_file(self, content: str, prompt_id: str) -> GitLabPromptTemplate:
if content.startswith("---"):
parts = content.split("---", 2)
if len(parts) >= 3:
frontmatter_str = parts[1].strip()
template_content = parts[2].strip()
else:
frontmatter_str = ""
template_content = content
else:
frontmatter_str = ""
template_content = content
metadata: Dict[str, Any] = {}
if frontmatter_str:
try:
import yaml
metadata = yaml.safe_load(frontmatter_str) or {}
except ImportError:
metadata = self._parse_yaml_basic(frontmatter_str)
except Exception:
metadata = {}
return GitLabPromptTemplate(
template_id=prompt_id,
content=template_content,
metadata=metadata,
)
def _parse_yaml_basic(self, yaml_str: str) -> Dict[str, Any]:
result: Dict[str, Any] = {}
for line in yaml_str.split("\n"):
line = line.strip()
if ":" in line and not line.startswith("#"):
key, value = line.split(":", 1)
key = key.strip()
value = value.strip()
if value.lower() in ["true", "false"]:
result[key] = value.lower() == "true"
elif value.isdigit():
result[key] = int(value)
elif value.replace(".", "").isdigit():
try:
result[key] = float(value)
except Exception:
result[key] = value
else:
result[key] = value.strip("\"'")
return result
def render_template(
self, template_id: str, variables: Optional[Dict[str, Any]] = None
) -> str:
if template_id not in self.prompts:
raise ValueError(f"Template '{template_id}' not found")
template = self.prompts[template_id]
jinja_template = self.jinja_env.from_string(template.content)
return jinja_template.render(**(variables or {}))
def get_template(self, template_id: str) -> Optional[GitLabPromptTemplate]:
return self.prompts.get(template_id)
def list_templates(self, *, recursive: bool = True) -> List[str]:
"""
List available prompt IDs under prompts_path (no extension).
Compatible with both list_files signatures:
- list_files(directory_path=..., file_extension=..., recursive=...)
- list_files(path=..., ref=None, recursive=...)
"""
# First try the "new" signature (directory_path/file_extension)
try:
files = self.gitlab_client.list_files(
directory_path=self.prompts_path,
file_extension=".prompt",
recursive=recursive,
)
base = self.prompts_path.strip("/")
out: List[str] = []
for p in files or []:
path = str(p).strip("/")
if base and not path.startswith(base + "/"):
# if the client returns extra files outside the folder, skip them
continue
if not path.endswith(".prompt"):
continue
out.append(self._repo_path_to_id(path))
return out
except TypeError:
# Fallback to the "classic" signature
raw = self.gitlab_client.list_files(
directory_path=self.prompts_path or "",
ref=None,
recursive=recursive,
)
# Classic returns GitLab tree entries; filter *.prompt blobs
files = []
for f in raw or []:
if (
isinstance(f, dict)
and f.get("type") == "blob"
and str(f.get("path", "")).endswith(".prompt")
and "path" in f
):
files.append(f["path"]) # type: ignore
return [self._repo_path_to_id(p) for p in files]
class GitLabPromptManager(CustomPromptManagement):
"""
GitLab prompt manager with folder support.
Example config:
gitlab_config = {
"project": "group/subgroup/repo",
"access_token": "glpat_***",
"tag": "v1.2.3", # optional; takes precedence
"branch": "main", # default fallback
"prompts_path": "prompts/chat"
}
"""
def __init__(
self,
gitlab_config: Dict[str, Any],
prompt_id: Optional[str] = None,
ref: Optional[str] = None, # tag/branch/SHA override
gitlab_client: Optional[GitLabClient] = None,
):
self.gitlab_config = gitlab_config
self.prompt_id = prompt_id
self._prompt_manager: Optional[GitLabTemplateManager] = None
self._ref_override = ref
self._injected_gitlab_client = gitlab_client
if self.prompt_id:
self._prompt_manager = GitLabTemplateManager(
gitlab_config=self.gitlab_config,
prompt_id=self.prompt_id,
ref=self._ref_override,
)
@property
def integration_name(self) -> str:
return "gitlab"
@property
def prompt_manager(self) -> GitLabTemplateManager:
if self._prompt_manager is None:
self._prompt_manager = GitLabTemplateManager(
gitlab_config=self.gitlab_config,
prompt_id=self.prompt_id,
ref=self._ref_override,
gitlab_client=self._injected_gitlab_client,
)
return self._prompt_manager
def get_prompt_template(
self,
prompt_id: str,
prompt_variables: Optional[Dict[str, Any]] = None,
*,
ref: Optional[str] = None,
) -> Tuple[str, Dict[str, Any]]:
if prompt_id not in self.prompt_manager.prompts:
self.prompt_manager._load_prompt_from_gitlab(prompt_id, ref=ref)
template = self.prompt_manager.get_template(prompt_id)
if not template:
raise ValueError(f"Prompt template '{prompt_id}' not found")
rendered_prompt = self.prompt_manager.render_template(
prompt_id, prompt_variables or {}
)
metadata = {
"model": template.model,
"temperature": template.temperature,
"max_tokens": template.max_tokens,
**template.optional_params,
}
return rendered_prompt, metadata
def pre_call_hook(
self,
user_id: Optional[str],
messages: List[AllMessageValues],
function_call: Optional[Union[Dict[str, Any], str]] = None,
litellm_params: Optional[Dict[str, Any]] = None,
prompt_id: Optional[str] = None,
prompt_variables: Optional[Dict[str, Any]] = None,
prompt_version: Optional[str] = None,
**kwargs,
) -> Tuple[List[AllMessageValues], Optional[Dict[str, Any]]]:
if not prompt_id:
return messages, litellm_params
try:
# Precedence: explicit prompt_version → per-call git_ref kwarg → manager override → config default
git_ref = prompt_version or kwargs.get("git_ref") or self._ref_override
rendered_prompt, prompt_metadata = self.get_prompt_template(
prompt_id, prompt_variables, ref=git_ref
)
parsed_messages = self._parse_prompt_to_messages(rendered_prompt)
if parsed_messages:
final_messages: List[AllMessageValues] = parsed_messages
else:
final_messages = [{"role": "user", "content": rendered_prompt}] + messages # type: ignore
if litellm_params is None:
litellm_params = {}
if prompt_metadata.get("model"):
litellm_params["model"] = prompt_metadata["model"]
for param in [
"temperature",
"max_tokens",
"top_p",
"frequency_penalty",
"presence_penalty",
]:
if param in prompt_metadata:
litellm_params[param] = prompt_metadata[param]
return final_messages, litellm_params
except Exception as e:
import litellm
litellm._logging.verbose_proxy_logger.error(
f"Error in GitLab prompt pre_call_hook: {e}"
)
return messages, litellm_params
def _parse_prompt_to_messages(self, prompt_content: str) -> List[AllMessageValues]:
messages: List[AllMessageValues] = []
lines = prompt_content.strip().split("\n")
current_role: Optional[str] = None
current_content: List[str] = []
for raw in lines:
line = raw.strip()
if not line:
continue
low = line.lower()
if low.startswith("system:"):
if current_role and current_content:
messages.append({"role": current_role, "content": "\n".join(current_content).strip()}) # type: ignore
current_role = "system"
current_content = [line[7:].strip()]
elif low.startswith("user:"):
if current_role and current_content:
messages.append({"role": current_role, "content": "\n".join(current_content).strip()}) # type: ignore
current_role = "user"
current_content = [line[5:].strip()]
elif low.startswith("assistant:"):
if current_role and current_content:
messages.append({"role": current_role, "content": "\n".join(current_content).strip()}) # type: ignore
current_role = "assistant"
current_content = [line[10:].strip()]
else:
current_content.append(line)
if current_role and current_content:
messages.append({"role": current_role, "content": "\n".join(current_content).strip()}) # type: ignore
if not messages and prompt_content.strip():
messages = [{"role": "user", "content": prompt_content.strip()}] # type: ignore
return messages
def post_call_hook(
self,
user_id: Optional[str],
response: Any,
input_messages: List[AllMessageValues],
function_call: Optional[Union[Dict[str, Any], str]] = None,
litellm_params: Optional[Dict[str, Any]] = None,
prompt_id: Optional[str] = None,
prompt_variables: Optional[Dict[str, Any]] = None,
**kwargs,
) -> Any:
return response
def get_available_prompts(self) -> List[str]:
"""
Return prompt IDs. Prefer already-loaded templates in memory to avoid
unnecessary network calls (and to make tests deterministic).
"""
ids = set(self.prompt_manager.prompts.keys())
try:
ids.update(self.prompt_manager.list_templates())
except Exception:
# If GitLab list fails (auth, network), still return what we've loaded.
pass
return sorted(ids)
def reload_prompts(self) -> None:
if self.prompt_id:
self._prompt_manager = None
_ = self.prompt_manager # trigger re-init/load
def should_run_prompt_management(
self,
prompt_id: Optional[str],
prompt_spec: Optional[PromptSpec],
dynamic_callback_params: StandardCallbackDynamicParams,
) -> bool:
return prompt_id is not None
def _compile_prompt_helper(
self,
prompt_id: Optional[str],
prompt_spec: Optional[PromptSpec],
prompt_variables: Optional[dict],
dynamic_callback_params: StandardCallbackDynamicParams,
prompt_label: Optional[str] = None,
prompt_version: Optional[int] = None,
) -> PromptManagementClient:
if prompt_id is None:
raise ValueError("prompt_id is required for GitLab prompt manager")
try:
decoded_id = decode_prompt_id(prompt_id)
if decoded_id not in self.prompt_manager.prompts:
git_ref = (
getattr(dynamic_callback_params, "extra", {}).get("git_ref")
if hasattr(dynamic_callback_params, "extra")
else None
)
self.prompt_manager._load_prompt_from_gitlab(decoded_id, ref=git_ref)
rendered_prompt, prompt_metadata = self.get_prompt_template(
prompt_id, prompt_variables
)
messages = self._parse_prompt_to_messages(rendered_prompt)
template_model = prompt_metadata.get("model")
optional_params: Dict[str, Any] = {}
for param in [
"temperature",
"max_tokens",
"top_p",
"frequency_penalty",
"presence_penalty",
]:
if param in prompt_metadata:
optional_params[param] = prompt_metadata[param]
return PromptManagementClient(
prompt_id=prompt_id,
prompt_template=messages,
prompt_template_model=template_model,
prompt_template_optional_params=optional_params,
completed_messages=None,
)
except Exception as e:
raise ValueError(f"Error compiling prompt '{prompt_id}': {e}")
async def async_compile_prompt_helper(
self,
prompt_id: Optional[str],
prompt_variables: Optional[dict],
dynamic_callback_params: StandardCallbackDynamicParams,
prompt_spec: Optional[PromptSpec] = None,
prompt_label: Optional[str] = None,
prompt_version: Optional[int] = None,
) -> PromptManagementClient:
"""
Async version of compile prompt helper. Since GitLab operations use sync client,
this simply delegates to the sync version.
"""
if prompt_id is None:
raise ValueError("prompt_id is required for GitLab prompt manager")
return self._compile_prompt_helper(
prompt_id=prompt_id,
prompt_spec=prompt_spec,
prompt_variables=prompt_variables,
dynamic_callback_params=dynamic_callback_params,
prompt_label=prompt_label,
prompt_version=prompt_version,
)
def get_chat_completion_prompt(
self,
model: str,
messages: List[AllMessageValues],
non_default_params: dict,
prompt_id: Optional[str],
prompt_variables: Optional[dict],
dynamic_callback_params: StandardCallbackDynamicParams,
prompt_spec: Optional[PromptSpec] = None,
prompt_label: Optional[str] = None,
prompt_version: Optional[int] = None,
ignore_prompt_manager_model: Optional[bool] = False,
ignore_prompt_manager_optional_params: Optional[bool] = False,
) -> Tuple[str, List[AllMessageValues], dict]:
return PromptManagementBase.get_chat_completion_prompt(
self,
model,
messages,
non_default_params,
prompt_id,
prompt_variables,
dynamic_callback_params,
prompt_spec=prompt_spec,
prompt_label=prompt_label,
prompt_version=prompt_version,
)
async def async_get_chat_completion_prompt(
self,
model: str,
messages: List[AllMessageValues],
non_default_params: dict,
prompt_id: Optional[str],
prompt_variables: Optional[dict],
dynamic_callback_params: StandardCallbackDynamicParams,
litellm_logging_obj: LiteLLMLoggingObj,
prompt_spec: Optional[PromptSpec] = None,
tools: Optional[List[Dict]] = None,
prompt_label: Optional[str] = None,
prompt_version: Optional[int] = None,
ignore_prompt_manager_model: Optional[bool] = False,
ignore_prompt_manager_optional_params: Optional[bool] = False,
) -> Tuple[str, List[AllMessageValues], dict]:
"""
Async version - delegates to PromptManagementBase async implementation.
"""
return await PromptManagementBase.async_get_chat_completion_prompt(
self,
model,
messages,
non_default_params,
prompt_id=prompt_id,
prompt_variables=prompt_variables,
litellm_logging_obj=litellm_logging_obj,
dynamic_callback_params=dynamic_callback_params,
prompt_spec=prompt_spec,
tools=tools,
prompt_label=prompt_label,
prompt_version=prompt_version,
ignore_prompt_manager_model=ignore_prompt_manager_model,
ignore_prompt_manager_optional_params=ignore_prompt_manager_optional_params,
)
class GitLabPromptCache:
"""
Cache all .prompt files from a GitLab repo into memory.
- Keys are the *repo file paths* (e.g. "prompts/chat/greet/hi.prompt")
mapped to JSON-like dicts containing content + metadata.
- Also exposes a by-ID view (ID == path relative to prompts_path without ".prompt",
e.g. "greet/hi").
Usage:
cfg = {
"project": "group/subgroup/repo",
"access_token": "glpat_***",
"prompts_path": "prompts/chat", # optional, can be empty for repo root
# "branch": "main", # default is "main"
# "tag": "v1.2.3", # takes precedence over branch
# "base_url": "https://gitlab.com/api/v4" # default
}
cache = GitLabPromptCache(cfg)
cache.load_all() # fetch + parse all .prompt files
print(cache.list_files()) # repo file paths
print(cache.list_ids()) # template IDs relative to prompts_path
prompt_json = cache.get_by_file("prompts/chat/greet/hi.prompt")
prompt_json2 = cache.get_by_id("greet/hi")
# If GitLab content changes and you want to refresh:
cache.reload() # re-scan and refresh all
"""
def __init__(
self,
gitlab_config: Dict[str, Any],
*,
ref: Optional[str] = None,
gitlab_client: Optional[GitLabClient] = None,
) -> None:
# Build a PromptManager (which internally builds TemplateManager + Client)
self.prompt_manager = GitLabPromptManager(
gitlab_config=gitlab_config,
prompt_id=None,
ref=ref,
gitlab_client=gitlab_client,
)
self.template_manager: GitLabTemplateManager = (
self.prompt_manager.prompt_manager
)
# In-memory stores
self._by_file: Dict[str, Dict[str, Any]] = {}
self._by_id: Dict[str, Dict[str, Any]] = {}
# -------------------------
# Public API
# -------------------------
def load_all(self, *, recursive: bool = True) -> Dict[str, Dict[str, Any]]:
"""
Scan GitLab for all .prompt files under prompts_path, load and parse each,
and return the mapping of repo file path -> JSON-like dict.
"""
ids = self.template_manager.list_templates(
recursive=recursive
) # IDs relative to prompts_path
for pid in ids:
# Ensure template is loaded into TemplateManager
if pid not in self.template_manager.prompts:
self.template_manager._load_prompt_from_gitlab(pid)
tmpl = self.template_manager.get_template(pid)
if tmpl is None:
# If something raced/failed, try once more
self.template_manager._load_prompt_from_gitlab(pid)
tmpl = self.template_manager.get_template(pid)
if tmpl is None:
continue
file_path = self.template_manager._id_to_repo_path(
pid
) # "prompts/chat/..../file.prompt"
entry = self._template_to_json(pid, tmpl)
self._by_file[file_path] = entry
# prefixed_id = pid if pid.startswith("gitlab::") else f"gitlab::{pid}"
encoded_id = encode_prompt_id(pid)
self._by_id[encoded_id] = entry
# self._by_id[pid] = entry
return self._by_id
def reload(self, *, recursive: bool = True) -> Dict[str, Dict[str, Any]]:
"""Clear the cache and re-load from GitLab."""
self._by_file.clear()
self._by_id.clear()
return self.load_all(recursive=recursive)
def list_files(self) -> List[str]:
"""Return the repo file paths currently cached."""
return list(self._by_file.keys())
def list_ids(self) -> List[str]:
"""Return the template IDs (relative to prompts_path, without extension) currently cached."""
return list(self._by_id.keys())
def get_by_file(self, file_path: str) -> Optional[Dict[str, Any]]:
"""Get a cached prompt JSON by repo file path."""
return self._by_file.get(file_path)
def get_by_id(self, prompt_id: str) -> Optional[Dict[str, Any]]:
"""Get a cached prompt JSON by prompt ID (relative to prompts_path)."""
if prompt_id in self._by_id:
return self._by_id[prompt_id]
# Try normalized forms
decoded = decode_prompt_id(prompt_id)
encoded = encode_prompt_id(decoded)
return self._by_id.get(encoded) or self._by_id.get(decoded)
# -------------------------
# Internals
# -------------------------
def _template_to_json(
self, prompt_id: str, tmpl: GitLabPromptTemplate
) -> Dict[str, Any]:
"""
Normalize a GitLabPromptTemplate into a JSON-like dict that is easy to serialize.
"""
# Safer copy of metadata (avoid accidental mutation)
md = dict(tmpl.metadata or {})
# Pull standard fields (also present in metadata sometimes)
model = tmpl.model
temperature = tmpl.temperature
max_tokens = tmpl.max_tokens
optional_params = dict(tmpl.optional_params or {})
return {
"id": prompt_id, # e.g. "greet/hi"
"path": self.template_manager._id_to_repo_path(
prompt_id
), # e.g. "prompts/chat/greet/hi.prompt"
"content": tmpl.content, # rendered content (without frontmatter)
"metadata": md, # parsed frontmatter
"model": model,
"temperature": temperature,
"max_tokens": max_tokens,
"optional_params": optional_params,
}