Practical prompt engineering for smaller LLMs  |  Articles  |  web.dev
Skip to main content
Resources
Web Platform
Dive into the web platform, at your pace.
HTML
CSS
JavaScript
User experience
Learn how to build better user experiences.
Performance
Accessibility
Identity
Learn
Get up to speed on web development.
Learn HTML
Learn CSS
Learn JavaScript
Learn AI
Learn Performance
Learn Accessibility
More courses
Additional resources
Explore content collections, patterns, and more.
AI and the web
Explore
PageSpeed Insights
Patterns
Podcasts & shows
Developer Newsletter
About web.dev
Discover
Baseline
How to use Baseline
Blog
Case Studies
/
English
Deutsch
Español – América Latina
Français
Indonesia
Italiano
Polski
Português – Brasil
Tiếng Việt
Türkçe
Русский
עברית
العربيّة
فارسی
हिंदी
বাংলা
ภาษาไทย
中文 – 简体
中文 – 繁體
日本語
한국어
Sign in
Articles
AI and the web
Fast load times
Learn Core Web Vitals
Identity
Progressive Web Apps
Payments
Notifications
How to optimize INP
Network reliability
React
Animations
Mini apps
Media
Safe and secure
WebAssembly
Devices
Easily discoverable
Test automation
Angular
Resources
More
AI and the web
Fast load times
Learn Core Web Vitals
Identity
Progressive Web Apps
Payments
Notifications
How to optimize INP
Network reliability
React
Animations
Mini apps
Media
Safe and secure
WebAssembly
Devices
Easily discoverable
Test automation
Angular
Discover
Baseline
How to use Baseline
Blog
Case Studies
Introduction
What is AI?
Ethics and AI
Right-sized AI and sustainability
Agents
Introduction to agents
Agent-friendly sites
Practical tips
Understand LLM size
Compare model capabilities
Choose a right-sized model
Improve performance and UX
Prompt engineering
Efficiently download AI models
Start building
Upgrade your site search
Combat toxicity
Use client-side AI to combat online toxicity
Build client-side AI toxicity detection
Improve product review suggestions
Built a local and offline-capable chatbot
Benefits and limits of large language models
Build a chatbot with WebLLM
Build a chatbot with the Prompt API
Resources
Learn AI
Watch to learn
The foundation model in Chrome
Gemini tutorials
Web Platform
HTML
CSS
JavaScript
User experience
Performance
Accessibility
Identity
Learn
Learn HTML
Learn CSS
Learn JavaScript
Learn AI
Learn Performance
Learn Accessibility
More courses
Additional resources
AI and the web
Explore
PageSpeed Insights
Patterns
Podcasts & shows
Developer Newsletter
About web.dev
Home
Articles
Resources
Practical prompt engineering for smaller LLMs
Stay organized with collections
Save and categorize content based on your preferences.
Maud Nalpas
GitHub
LinkedIn
Bluesky
A large language model's effectiveness heavily relies on the instructions we
give it. Prompt engineering is the
process of crafting questions in a way that gets the best output from an LLM.
It's a crucial step in implementing an LLM-based feature.
Prompt engineering is an iterative process. If you've experimented with
different LLMs, you've probably noticed that you needed to tweak your prompt to
achieve a better result.
This is also true for models of different sizes.
Chat interfaces powered by large LLMs, such as Gemini
or ChatGPT, can often produce satisfying results with minimal prompting effort.
However, when working with a default, smaller LLM that is not fine tuned, you
need to adapt your approach.
Smaller LLMs are less powerful and have a smaller pool of information to draw from.
What do we mean with "smaller LLMs"?
Defining LLM sizes is complicated, and they aren't always
disclosed by the makers.
In this document, "smaller LLMs" means any model under 30B parameters.
As of today, models with a few millions to a few billions parameters can
realistically be run in the browser, on consumer-grade devices.
Where are smaller LLMs used?
On-device/in-browser generative AI, for example if you're using Gemma 2B
with MediaPipe's LLM Inference API
(even suitable for CPU-only devices) or DistilBert in the browser with
Transformers.js.
Downloading a model and running inference on a user's device is only possible
with these smaller LLMs, in order to keep web downloads reasonable and to suit
a device's memory and GPU/CPU constraints.
Custom server-side generative AI. Small open-weight models like Gemma 2B,
Gemma 7B or Gemma 27B are available for you to run on your own server (and
optionally fine tune).
Get started
Note: It's useful to document the evolution of a prompt when prompt engineering.
This helps show others what's already been done when trying to improve the
prompt. For example, you could copy-paste the prompts attempted into a doc, with
notes on the changes.
Chat interfaces powered by large LLMs, like Gemini or ChatGPT, can often produce satisfying results with minimal prompting effort. However, when working with smaller LLMs, you need to adapt your approach. Smaller LLMs are less powerful and have a smaller pool of information to draw from.
Provide context and precise format instructions
To achieve optimal results with small LLMs, craft more detailed and specific prompts.
For example:
Based on a user review, provide a product rating as an integer between 1 and 5. \n
Only output the integer.
Review: "${review}"
Rating:
Note: The formatting variations you see are not errors. They reflect the way the model generates responses.
Input (review)
Output (rating)
 
Larger LLM (Gemini 1.5)
Smaller LLM (Gemma 2B)
Absolutely love the fit! Distributes weight well and surprisingly comfortable even on all-day treks. Would recommend.
5
4 out of 5 stars**
The straps are flimsy, and they started digging into my shoulders under heavy loads.
1
2/5
While Gemini 1.5 provides the desired output with this simple prompt, Gemma's
output isn't satisfying:
The format is incorrect. We requested an integer for the rating.
The rating doesn't seem quite accurate. The first review is enthusiastic enough to indicate a 5-star review.
To fix this, we need to use prompt engineering techniques,
One-, few-, and multi-shot prompting
and chain-of-thought prompting.
We also must provide clear format instructions and insist that the model should
use the full range of ratings.
For example:
`Analyze a product review, and then based on your analysis give me the
corresponding rating (integer). The rating should be an integer between 1 and
5. 1 is the worst rating, and 5 is the best rating. A strongly dissatisfied
review that only mentions issues should have a rating of 1 (worst). A strongly
satisfied review that only mentions positives and upsides should have a rating
of 5 (best). Be opinionated. Use the full range of possible ratings (1 to
5). \n\n
\n\n
Here are some examples of reviews and their corresponding analyses and
ratings:
\n\n
Review: 'Stylish and functional. Not sure how it'll handle rugged outdoor
use, but it's perfect for urban exploring.'
Analysis: The reviewer appreciates the product's style and basic
functionality. They express some uncertainty about its ruggedness but
overall find it suitable for their intended use, resulting in a positive,
but not top-tier rating.
Rating (integer): 4
\n\n
Review: 'It's a solid backpack at a decent price. Does the job, but nothing
particularly amazing about it.'
Analysis: This reflects an average opinion. The backpack is functional and
fulfills its essential purpose. However, the reviewer finds it unremarkable
and lacking any standout features deserving of higher praise.
Rating (integer): 3
\n\n
Review: 'The waist belt broke on my first trip! Customer service was unresponsive too. Would not recommend.'
Analysis: A serious product defect and poor customer service experience naturally warrants the lowest possible rating. The reviewer is extremely unsatisfied with both the product and the company.
Rating (integer): 1
\n\n
Review: 'Love how many pockets and compartments it has. Keeps everything organized on long trips. Durable too!'
Analysis: The enthusiastic review highlights specific features the user loves (organization and durability), indicating great satisfaction with the product. This justifies the highest rating.
Rating (integer): 5
\n\n
Review: 'The straps are a bit flimsy, and they started digging into my shoulders under heavy loads.'
Analysis: While not a totally negative review, a significant comfort issue leads the reviewer to rate the product poorly. The straps are a key component of a backpack, and their failure to perform well under load is a major flaw.
Rating (integer): 1
\n\n
Now, here is the review you need to assess:
\n
Review: "${review}" \n`;
Note: The formatting variations you see are not errors. They reflect the way the
model generates responses.
Input (review)
Output with a smaller LLM (Gemma 2B)
Absolutely love the fit! Distributes weight well and surprisingly
comfortable even on all-day treks. Would recommend.
Analysis: This review is highly positive and emphasizes the product's
fit and comfort, indicating great satisfaction with the product. This
justifies a high rating.
Rating (integer): 5
The straps are flimsy, and they started digging into my shoulders
under heavy loads.
**Analysis:** The review highlights a significant comfort issue with
the straps. It also suggests that the product may not be suitable for heavy
loads or prolonged outings.
**Rating:** 1 (worst)
Our more elaborate prompt, combined with few-shot prompting and chain-of-thought prompting techniques, gives more accurate ratings.
Expect manual output parsing
While you should always manually clean up and interpret an LLM's output, this is
particularly true for smaller LLMs which might produce less polished results and
may require chain-of-thought prompting.
In the first example, we used chain-of-thought prompting, so the output included
both an analysis and a rating and we need to parse manually for the rating. Also
note the inconsistent formatting in the output it the previous section: the model sometimes outputs markdown, but not every time.
// Use try/catch to catch (parsing) errors
try {
// Parse the output for a rating
const int = /[1-5]/;
const ratingAsString = llmOutput.match(int)[0];
// Parse as int
const ratingAsInt = parseInt(ratingAsString);
// `checkRating` checks the rating is a number between 1 and 5, since the
// regEx may catch a number like "100"
const finalRating = checkRating(ratingAsInt);
} catch (e) {
console.error('Error', e);
}
Mind API differences
LLM cloud APIs like the Gemini API or OpenAI, which are typically the entry
point to larger LLMs, offer handy prompt features. For example, Gemini 1.5 Pro
offers system instructions and
JSON mode.
At the moment, these features aren't always available for custom model usage, or
for smaller LLMs accessed using in-browser AI APIs, such as the
MediaPipe LLM Inference API
or Transformers.js.
While this isn't necessarily a technical limitation, in-browser AI APIs tend to
be leaner.
Mind token limits
Because your prompt for smaller LLMs needs to include examples or more detailed
instructions, it will likely be longer and take up more of your input token
limit, if there is one.
Additionally, smaller models tend to have a smaller input token limit. For
example, Gemini 1.5 Pro has a 1-million input token limit
while Gemma models have a 8K context window.
Use token count functions
to avoid hitting the limit.
Adapt your times estimates
Account for prompt design and testing in your engineering time estimates.
Due to API differences and token limits, you'll likely need more time and effort
to craft your prompt for a smaller LLM than a larger one. Testing and validating
the LLM's output may also be higher effort.
Prompt engineer versus fine tuning?
For web developers, prompt engineering is our preferred way of leveraging
generative AI
over custom training and fine-tuning.
But even advanced prompt engineering may not be sufficient in some use cases,
especially if you're using a smaller LLM.
Use fine-tuning when:
You require top-notch accuracy and performance for a specific task. Fine
tuning directly adjusts the model's internal parameters for optimal results.
You have well-curated data, relevant to your task, already labeled with
preferred outputs. You need this data for effective fine-tuning.
You use the model for the same purpose repeatedly. Fine tuning can be done
once, and reused for a specific task.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-05-30 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-05-30 UTC."],[],[]]
web.dev
web.dev
We want to help you build beautiful, accessible, fast, and secure websites that work cross-browser, and for all of your users. This site is our home for content to help you on that journey, written by members of the Chrome team, and external experts.
Contribute
File a bug
See open issues
Related Content
Chrome for Developers
Chromium updates
Case studies
Podcasts & shows
Follow
@ChromiumDev on X
YouTube
Chrome for Developers on LinkedIn
RSS
Terms
Privacy
Manage cookies
English
Deutsch
Español – América Latina
Français
Indonesia
Italiano
Polski
Português – Brasil
Tiếng Việt
Türkçe
Русский
עברית
العربيّة
فارسی
हिंदी
বাংলা
ภาษาไทย
中文 – 简体
中文 – 繁體
日本語
한국어