We love fine-tuning, but if we're being honest, not every use case needs a fine-tuned model! Prompt engineering a one or few-shot response is good enough in a bunch of cases.

This begs the question, so when should you fine-tune a model?

Let's look at a very common use for LLMs, tweeting.

Cost of a Tweet

Whether or not you are a premium user, the above-the-fold length of your tweets is going to max out at 280 characters. This means that a tweet is going to be at most 40-50 words on average, with a real life average of 20-30 words. For the purposes of this example, we will assume a tweet length of 35 words, or 46 tokens per tweet (assuming an industry standard average of .75 words per token)

Prompting Base Cost

The base cost of simple prompting is the cheapest way to utilize LLMs. The cost is simple and easily predictable.

Prompt: Write an engaging tweet about the speed of LLM text generation vs human speech (spoiler: LLMs are faster than people)

When you just need content, the cost of prompting is minimal. If you don't care about tone, expertise, or rule enforcement, you can get away with a simple prompt and a few-shot example or two.

Prompting Hidden Costs

In many cases, you do not just want whatever output the LLM gives you by default. In our example of crafting a tweet, there are a bunch of factors to consider. From a holistic standpoint, tone and expertise are important when curating tweets for your account. For platform considerations, only the first 280 characters are shown on your viewers' timelines, and you may want to include hashtags to reach your target audience.

When taking all of these into account, the costs of prompt engineering grow quickly

You are an experienced social media content strategist working within the communications department of a prestigious, forward-thinking AI research institution. Your primary objective is to craft compelling and shareable content that demystifies complex technological advancements for a broad audience. Specifically, you need to generate a tweet that is both highly informative and genuinely engaging, designed to pique the curiosity of both tech-savvy individuals and the general public.

The core message of this tweet is to highlight the extraordinary speed of Large Language Model (LLM) text generation. To make this point relatable and impactful, you will draw a direct comparison to the average pace of human speech. It is crucial to emphasize, without being overly technical or dry, that LLMs are demonstrably faster at producing text than humans are at speaking. Think of this as a "wow factor" revelation.

• Tone: The overall tone should be enthusiastic, optimistic, and slightly awe-inspiring, while maintaining a foundation of professionalism. It should feel friendly and approachable, like a knowledgeable expert sharing exciting news, not a rigid academic paper. Avoid jargon where simpler language can be used, but don't shy away from using terms like "LLM" if explained implicitly by context.
• Narrative Arc (Implicit): Briefly set the scene, introduce the comparison, reveal the "spoiler" (LLMs are faster), and end with a forward-looking or thought-provoking statement.
• Conciseness & Impact: While the output needs to be a tweet (max 280 characters), your prompt is designed to guide the AI to think about more, ensuring the final tweet is impactful within its constraints. The ultimate goal is a tweet that sparks conversation and encourages retweets or likes.
• Hashtags: Integrate a selection of relevant and trending hashtags organically within the tweet or at the end. Consider a mix of broad terms and more specific ones. Examples to consider include: #AI #LLM #TechInnovation #FutureOfAI #GenerativeAI #ArtificialIntelligence #SpeedOfTech. Aim for 3-5 relevant hashtags.
• Call to Action (Subtle): While not an explicit "click here" CTA, the tweet should subtly encourage engagement, perhaps by posing a rhetorical question or a thought-provoking statement that invites replies or shares.
• Emotional Appeal: Evoke a sense of wonder and excitement about the rapid advancements in AI. The goal is to make people feel intrigued, not intimidated.
• Originality: The tweet should feel fresh and avoid generic phrasing. Think about how to phrase the speed comparison in a memorable way.
• Constraints (for the final tweet):
  • Maximum 280 characters.
  • No more than two emojis (optional, if they enhance the friendly tone without detracting from professionalism).
  • No external links.
Consider these additional factors when generating the tweet:
• Audience Empathy: Think about how the average Twitter user might react to this information. How can you make it digestible and exciting for them?
• Institutional Voice: Ensure the tweet subtly reflects the innovative and credible nature of an AI research institution. It should inspire trust.
• Avoiding Hyperbole: While exciting, avoid making claims that sound unrealistic or overly sensationalized. Stick to the core fact about comparative speed.

Based on these comprehensive guidelines, craft a tweet that captures the essence of LLM speed versus human speech in a captivating, professional, and friendly manner.

Fine-Tuning Costs

The nice thing about fine-tuning costs is, that while the rates might be higher than the rates of the base model, the costs are once again fixed and predictable. With your fine-tuned model, your prompts can be pared back down to exactly what you need. This means we can return to our original prompt and calculate costs from there.

Conclusion

Our long prompting example may have been a little verbose, but it is indeed well within the bounds of prompts that you will see in a production use of AI. In our use case, for the GPT-4.1-mini model, assuming a fixed length output, the fine-tuned model becomes profitable when the base prompt needs to be extended to ~250 tokens from the 27 token base prompt.

Considerations

There are a few important things of note about this example worth paying attention to:

• Completion Length: As you may have picked up on, output/completion tokens are much more expensive than input tokens. On average, output tokens are about 4x the cost of an input. That means that when you are considering a fine-tuned model, if you will be using a high volume of prompts for short completions, you may see significant cost savings by fine-tuning your own. If you are prompting outputs that are much longer than the the prompts themselves, it may benefit you to stick with prompt engineering.
• Model Selection: So far, we have made the assumption that you are going to be using the same model across all of your uses. This is however definitely not always the case. In many cases, fine-tuning a cheaper model for a specific use case can actually be as effective or even more effective than a more expensive model for that use case. As an example, an fine-tuned model based on GPT-4.1-mini has a price-point of $0.80 / 1M input, $3.20 / 1M output, while the base version of GPT-4.1 is a much higher $2.00 / 1M input, $8.00 / 1M output. In this case, if you have been doing a task that requires the larger frontier model, you may see significant cost savings of 60% by fine-tuning your own. If you are working on a task like classification you may be able to move down to an even cheaper model still.
• Behavior: Removed from cost, there are other considerations. Sometimes it is hard to verbalize exactly the behavior you need out of your model. For other tasks, you just need to provide the model with deeper domain knowledge than you can realistically provide in a prompt with every request. In these cases, showing your model via fine-tuning can provide the results you need, and the cost may just be a secondary benefit.