feedback_forensics.tools.ff_modelgen

Tools to generate responses from models for comparison of model personalities.

Module Contents

Functions

run_model_on_prompts
run_model_on_prompts_async	Process prompts concurrently using asyncio with controlled concurrency.
load_generations	Load generations from output path.

API

feedback_forensics.tools.ff_modelgen.run_model_on_prompts(prompts: list[str], model_name: str, output_path: str)

async feedback_forensics.tools.ff_modelgen.run_model_on_prompts_async(prompts: List[str], model_name: str, output_path: str, max_concurrent: int = 5, max_tokens: int = 10000)

Process prompts concurrently using asyncio with controlled concurrency.

Args: prompts: List of prompts to process model_name: Name of the model to use output_path: Directory to save results max_concurrent: Maximum number of concurrent tasks

Example:

import asyncio

# Call the async function with asyncio.run
asyncio.run(run_model_on_prompts_async(
    prompts=your_prompts,
    model_name="your-model",
    output_path="output/directory",
    max_concurrent=10  # Adjust based on your needs
))

feedback_forensics.tools.ff_modelgen.load_generations(output_path: str, model_name: str)

Load generations from output path.