SLM Builder
A comprehensive Python toolkit for creating, training, and deploying specialized language models with support for multiple data sources, training methods, and deployment formats.
Version 1.0.0 - Production Ready
Quick Start
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from slm_builder import SLMBuilder
# Create a QA model
builder = SLMBuilder(
project_name="my-qa-model",
base_model="google/flan-t5-small"
)
# Load data and train
builder.load_data_from_csv("qa_data.csv")
builder.train(epochs=3, learning_rate=2e-5)
# Evaluate and export
metrics = builder.evaluate()
builder.export_model("./output", format="huggingface")
📚 Documentation
Getting Started
| Guide | Description |
|---|---|
| Installation | Complete installation instructions |
| Quick Start | Build your first model in 5 minutes |
| Basic Concepts | Core concepts and architecture |
| Quick Reference | Cheat sheet for common tasks |
Features
| Feature | Description |
|---|---|
| Model Loading | Load from HuggingFace, local, Ollama, GGUF |
| Data Sources | 14 data sources: SQL, MongoDB, APIs, files |
| Training | LoRA, QLoRA, full fine-tuning |
| Evaluation | Comprehensive metrics and benchmarks |
| Quantization | 4-bit and 8-bit compression |
| Model Comparison | Benchmark multiple models |
| Export | ONNX, TorchScript, HuggingFace |
Examples
| Category | Examples |
|---|---|
| Getting Started | Basic usage patterns |
| Database Integration | PostgreSQL, MySQL, MongoDB, Redis |
| API Integration | REST APIs, authenticated endpoints |
| Advanced | Production use cases |
Reference
| Document | Description |
|---|---|
| API Reference | Complete API documentation |
| Configuration | All configuration options |
| CLI Reference | Command-line interface |
Development
| Guide | Description |
|---|---|
| Developer Guide | Contributing and development |
| Changelog | Version history |
✨ Key Features
🎯 14 Data Sources
CSV, JSON, JSONL, Parquet, SQL (PostgreSQL, MySQL, SQLite), MongoDB, Redis, Elasticsearch, REST APIs, and more.
🚀 Flexible Training
LoRA fine-tuning, QLoRA (4-bit), full fine-tuning, custom configurations.
📊 Advanced Evaluation
Perplexity, BLEU, ROUGE, F1, accuracy, and task-specific metrics.
⚡ Optimized Performance
Quantization, GPU/CPU support, batch processing, efficient inference.
📦 Multiple Exports
HuggingFace Hub, ONNX, TorchScript, TensorFlow, local deployment.
🔧 Production Ready
MLflow tracking, dataset splitting, model comparison, comprehensive logging.
🎯 Use Cases
Customer Support Bot
Build an intelligent QA bot from support tickets:
1
2
3
4
builder.load_from_database(
connection_params={"dialect": "postgresql"},
query="SELECT question, answer FROM support_tickets"
)
Document Intelligence
Extract insights from document collections:
1
2
3
4
builder.load_from_elasticsearch(
host="localhost:9200",
index="documents"
)
API-Driven Models
Train on data from REST APIs:
1
2
3
4
builder.load_from_api(
endpoint="https://api.example.com/data",
headers={"Authorization": "Bearer TOKEN"}
)
📦 Installation
1
2
3
4
5
6
7
8
# Basic installation
pip install slm-builder
# With database support
pip install slm-builder[db]
# With all features
pip install slm-builder[all]
See the Installation Guide for detailed instructions.
🤝 Community & Support
Get Help
- GitHub Issues - Report bugs
- Discussions - Ask questions
- Documentation - Guides and references
Contribute
We welcome contributions! See our Developer Guide to get started.
📊 Project Status
Last Updated: December 2, 2025
License
SLM Builder is released under the MIT License. See LICENSE for details.