Design an LLM-based binary classifier

You are given a helper function score_batch(prompt, inputs) that, for each input string, returns per-token probabilities (possibly as log-probabilities) from a large language model under the provided system prompt. Design a binary text classifier using only this helper: ( 1) Construct system prompts for classes A and B. ( 2) For each input, query the helper with both prompts and produce a continuous score s in [0,1] estimating P(class=A|x), not just a hard label. ( 3) If the helper returns log-probabilities, show how to compute a numerically stable score (e.g., via log-sum-exp and log-prob normalization). ( 4) Define batching and how token-level probabilities are aggregated into a single sequence-level score. ( 5) Describe threshold selection and evaluation metrics (ROC-AUC, precision/recall, F 1), including calibration. ( 6) Propose performance improvements: prompt engineering, repeated sampling, temperature control, prompt ensembling, score calibration, and handling class imbalance. ( 7) Discuss failure modes and how to validate/iterate offline without fine-tuning. Provide pseudocode and the inference-time complexity.