⚙️ Inference Engines¶
Powerful Model Execution Engines
Synchronous and asynchronous inference with advanced features
🚀 Engine Types¶
Synchronous Engine¶
from llm_evaluation_framework.model_inference_engine import ModelInferenceEngine
engine = ModelInferenceEngine(registry)
results = engine.evaluate_model(model_name, test_cases, use_case)
Asynchronous Engine¶
from llm_evaluation_framework.engines.async_inference_engine import AsyncInferenceEngine
async_engine = AsyncInferenceEngine(registry)
results = await async_engine.evaluate_model_async(model_name, test_cases, use_case)
📊 Performance Optimization¶
- Batch Processing: Process multiple requests efficiently
- Concurrent Execution: Run async evaluations
- Caching: Cache results for repeated queries
- Rate Limiting: Respect API limits