Struggle of LLMs with Merged Instructions
When One Instruction Tries to Do Too Much
Large Language Models (LLMs) have revolutionized text generation, coding assistance, and even creative writing. However, despite their impressive capabilities, they have significant limitations—particularly in understanding and executing multiple system instructions when they are merged into a single statement.
In this post, we'll explore why LLMs struggle with these complex instructions and why they require precise, pinpointed directives for optimal performance.
LLMs Process Instructions Linearly, Not Holistically
At their core, LLMs generate text based on probability, predicting the next word given previous context. Unlike a human who can comprehend multiple instructions simultaneously and infer intent, LLMs process text in a sequential manner.
When multiple system prompts or directives are merged into a single instruction, LLMs can misinterpret, overlook, or improperly prioritize components of the command. They end up applying parts of the instruction incorrectly or missing critical elements entirely.
Example 1: A Simple Merged Instruction That Fails
Consider the following instruction:
Merged Command:
"Summarize this article, extract key points, and rewrite them in bullet points along with a brief conclusion."
What can go wrong?
The LLM might correctly summarize the article but fail to extract distinct key points.
It might include bullet points but forget to add the conclusion.
The conclusion might be generic because the model didn’t treat it as a separate section.
In this scenario, the model’s sequential processing causes it to emphasize certain tasks over others, resulting in an incomplete response.
Example 2: A Complex Merged Instruction That we were trying at Mailmodo
Now look at the following Merged Instruction that we were trying with ChatGPT Model 4o.
Create new color configurations for HTML email templates based on the input JSON to improve readability, accessibility, and visual appeal. The configurations should comply with specific guidelines, respect component hierarchy (z-index), offer flexible configuration options, and be adaptable based on user feedback.
Steps to follow in generating color scheme
1. Analyze the Input Configuration
a. The input JSON includes:
i. Blocks: Represented by indices in the path field (e.g., blocks[0], blocks[1]).
ii. UI Components: Defined using dot-delimited formats (e.g., properties.button1.value.borderColor).
iii. Inline Styles: Provide additional styling details (e.g., background-color or color for specific text elements).
b. Examples:
i. "blocks[4].properties.background1.value.backgroundColor2" refers to the second background color of background1 in block 5.
ii. "blocks[1].properties.button1.value.color" indicates the text color of button1 in block 2.
2. Respect Component Hierarchy (Z-Index)
a. Each block has a block background at the lowest z-index.
b. Card components within a block appear at a higher z-index than the block background but lower than individual elements placed on the card.
c. Elements such as text, images, buttons, menu items, & social media icons appear at the top.
d. When assigning or updating colors, ensure that the hierarchy is respected. For instance, do not reuse a block’s background color for the foreground of a higher z-index element if it compromises contrast or readability.
e. Use the information in the path field to preserve interdependencies among components.
3. Specify Customization Options for Each Component
a. Block: Background color(s), border color, border width, padding, etc.
b. Card: Background color, border color, border width, border radius, etc.
c. Button: Fill color, text color, border color, border radius, etc.
d. Text Elements: Text color, link color, background color, etc.
e. Images: Border color, frame styling.
f. Social Media icons: Icon color, Icon shape, fill color for the element, etc.
4. Guidelines to apply color to the template
a. Use the provided color palette for the email template in user message
i. primaryColorOne (Saturated Color): A bold, attention-grabbing color ideal for driving calls-to-action or highlighting key focal points.
ii. primaryColorTwo (Lighter Color): A softer, de-saturated version of the primary color, used for less aggressive accents and section headers.
iii. secondaryColorOne (Saturated Color): A vibrant, complementary hue that adds contrast and emphasis to secondary sections or visuals.
iv. secondaryColorTwo (Lighter Color): A muted, pastel version of the secondary color, perfect for soft backgrounds, subtle highlights, or visual balance.
v. neutralColor : A foundational tone (light or dark) that provides a clean, consistent backdrop for content and balances the bold accents.
b. Theme Determination
i. The theme is decided based on the Neutral color:
a. Light Theme : If Neutral is light, use light backgrounds with dark text.
b. Dark Theme : If Neutral is dark, use dark backgrounds with light text.
c. Light Theme Behavior
i. Headers/Banners
Default Background :
Use white (#FFFFFF) or Neutral for clean, minimal designs.
For Emphasis :
Use Primary One or Secondary One in bold/saturated tones for prominent branding.
Text Color :
On light headers: Use black (#000000).
On bold headers: Use white (#FFFFFF).
ii. Buttons/Call-to-Actions (CTAs)
i. Use Primary One or Secondary One for visibility and differentiating them from surrounding text or backgrounds.
ii. Text Contrast :
Use black (#000000) on light buttons.
Use white (#FFFFFF) on dark buttons.
iii. Section Backgrounds
Use Primary Two , Secondary Two, Neutral , or white (#FFFFFF) for structure and balance.
Use Subtle Variations in background colors to differentiate sections and guide the reader’s eye flow.
Text Overlay :
Use black (#000000) or dark gray (#333333) on light backgrounds.
iv. Text and Titles
General Text Colors :
Use black (#000000) or dark gray (#333333) for primary text and titles.
Use gray (#808080) for placeholder, metadata, or less prominent text.
Highlighting Text :
For important or prominent text elements, use Primary One or Secondary One to draw attention.
Ensure text colors provide clear contrast against the background (e.g., Primary One text on white must still be readable).
v. Decorative Elements
Use Neutral, Primary Two or Secondary Two for subtler visuals (e.g., separators, highlights).
d. Dark Theme Behavior
i. Headers/Banners
Default Background :
Use Neutral or Primary One/Secondary One in darker tones to define structure.
Text Color :
On dark headers: Use white (#FFFFFF).
On light/vibrant headers: Use black (#000000).
ii. Buttons/Call-to-Actions (CTAs)
Use Primary Two, Secondary Two, or Neutral for visibility and differentiating them from surrounding text or backgrounds..
Text Contrast :
Use black (#000000) on light buttons.
Use white (#FFFFFF) on dark buttons.
iii. Section Backgrounds
Select Neutral , Primary One, or Secondary Two for structure and balance.
Use Subtle Variations in background colors to differentiate sections and guide the reader’s eye flow.
Text Overlay :
Use white (#FFFFFF) or light gray (#EEEEEE) on dark backgrounds.
iv. Text and Titles
General Text Colors :
Use white (#FFFFFF) or light gray (#EEEEEE) for primary text and titles.
Use gray (#808080) for placeholder, metadata, or less prominent text.
Highlighting Text :
Use Primary One or Secondary One to emphasize key text elements, ensuring strong contrast with the background.
v. Decorative Elements
Use Secondary Two or Primary Two for subtler visuals (e.g., separators, highlights).
e. Accessibility and Contrast
Contrast Standards
Adhere to WCAG guidelines:
Body text: Minimum contrast ratio of 4.5:1 .
Large text (18pt+): Minimum contrast ratio of 3:1 .
Fallback for Clashes
Replace clashing foreground/background combinations with Neutral or de-saturated versions of Primary/Secondary colors .
For readability, adjust text colors to black , white , or gray.
f. Maintain a Consistent Scheme and Color Harmony
i. For each section (aka block), decide the background colors, card colors, and elements styling based on the layout used (e.g., whether it is primarily text, a combination of text and images, or CTA-heavy) or type of content in the section (e.g. feature highlight, customer testimonial, etc.)
ii. Ensure each section stands out enough from adjacent sections without breaking the overall flow.
iii. Avoid visual clutter by using consistent colors for repeated elements throughout the template.
iv. Leverage Psychological Effects of Colors to align with the email’s campaign objectives.
v. Optimize for Dark Mode. For e.g., avoid light gray text on dark backgrounds.
g. Creative Flexibility for AI
a. Use these color guidelines as a general framework while exercising creative decision-making to meet the design's purpose and branding.
b. Adapt Primary One and Secondary One dynamically to highlight specific text elements or components where prominence is required.
c. Combine contrast, saturation, and layering techniques to create a harmonious look that adheres to both aesthetics and accessibility.
5. Maintain Input Structure
a. Keep the structure of the input JSON unchanged.
b. Modify only the color values within the colorData structure.
6. Output Format Guidelines
a. Deliver the results in the following JSON format:
{
"colorData": [
{
"path": "blocks[0].properties.background1.value.backgroundColor",
"color": "newColorValue"
},
{
"path": "blocks[0].properties.border1.value.borderColor",
"color": "newColorValue"
},
{
"path": "blocks[1].properties.button1.value.color",
"color": "newColorValue"
},
...
]
}
b. Populate the colorData array with only updated color values
c. Do not include additional details or commentary beyond the points listed above.`;The results were good and adhering 80% of the time with temperature value of 1. But 80% is not good enough. We want more robustness and more certainity. You can see this instruction is attempting to do multiple things at once:
Modify JSON-based color values.
Ensure improvements in readability, accessibility, and visual appeal.
Comply with multiple guidelines covering color hierarchy, contrast, and theme consistency.
Allow configuration flexibility while maintaining user adaptability.
Given the broadness of this instruction, the LLM was doing one of following in different iterations:
Successfully update colors but fail to follow z-index rules.
Ensure visual appeal but ignore accessibility contrast checks.
Generate an output JSON file but fail to preserve the original input structure.
Breaking Down the Merged Instruction into Smaller, Targeted Prompts
To improve performance, let’s split the complex instruction into separate, smaller system prompts, ensuring each aspect is handled properly.
Step 1: Isolate the Task of Updating Color Configurations
🔹 System Instruction:
"Modify the color values in the provided JSON structure while preserving its format. Do not change non-color fields."
This keeps the focus only on updating colors while ensuring that JSON integrity is maintained.
Step 2: Ensure Accessibility and Readability
🔹 System Instruction:
"Apply WCAG-compliant contrast ratios to ensure text remains readable against its background. Maintain a minimum contrast ratio of 4.5:1 for body text and 3:1 for large text."
This extracts the accessibility rule from the original large instruction, making it explicit and ensuring the model prioritizes it correctly.
Step 3: Implement Component Hierarchy (z-index Consideration)
🔹 System Instruction:
"Ensure that color assignments respect the component hierarchy (z-index). Background colors should be distinct from foreground elements, and high-z-index elements such as buttons and text should have sufficient contrast from their backgrounds."
By specifying z-index behavior separately, the LLM can properly handle layering and precedence of UI components.
Step 4: Apply Theme Guidelines
🔥 System Instruction for Light & Dark Themes:
"Determine the appropriate theme based on the neutral color. If the neutral tone is light, apply a light theme using dark text on light backgrounds. If the neutral tone is dark, use a dark theme with light text on dark backgrounds."
This guarantees that theme determination follows a systematic decision-making process, rather than getting lost in a broad prompt.
Step 5: Format the Output Correctly
📌 System Instruction:
"Generate the final output in JSON format, including only modified color values within the
colorDataarray. Retain the originalpathfield structure and do not add extra data."
This scope clarification prevents the LLM from including unnecessary explanations in the output.
Fundamental of Software Engineering is retained
The idea is to imagine the whole system as very small unit level assistant working together in coherence to achive the common goal. Its very similar to writing small unit testable function while developing large softwares.
Frameworks that help achieve this
And here where you can use framework like Langchain or Langgraph to think about architecture that suites the best for your usecase and then orchestrate the entire flow using the framework to make sure your small little assistants are doing exactly what they are supposed to do to increase the accuracty and predictibility of the results.
The architecture of the workflow can looks like as follows:-
All nodes in the above workflows are small small assistants doing pin pointed things and no complex/merged instruction in any of the node above.
Why Breaking the Prompt Into Steps Improves Results
When LLMs process a highly structured sequence of targeted instructions rather than a single overloaded command , they:
✔ Follow each guideline without skipping essential details.
✔ Prioritize essential constraints like accessibility and hierarchy.
✔ Maintain JSON structure without unintentionally modifying non-color data.
✔ Ensure consistency in theme-based color applications.
By separating concerns across structured system prompts, we eliminate ambiguity and enhance task execution quality.
Many LLM failures arise when multiple objectives are crammed into a single instruction. By decomposing complex tasks into smaller, specific instructions, we significantly improve the AI’s ability to generate the expected result accurately.
Next time you write a task for an AI model, try splitting it into discrete, focused prompts —the results will speak for themselves!