There is a particular kind of client request that sounds simple and is not: put the AI where our customers already are. No app to install, no portal to log into. Just WhatsApp.
I built one of these — an interior design assistant that runs entirely inside WhatsApp. A user sends a floor plan as a photo or a PDF, describes the style they want by text, image or voice, and gets back a written design concept, photorealistic room-by-room renders in one cohesive style, and a marked-up blueprint of the redesigned area. Then they refine it in plain language: make the retail area bigger, try it in a warmer palette. The agent redesigns on the spot.
Most of the engineering is not the AI part.
Everything arrives as a different shape
A chat interface on the web gives you a text box. WhatsApp gives you whatever the person felt like sending. A photo of a floor plan. A PDF from an architect. A thirty-second voice note describing a kitchen. Sometimes all three across four messages.
So before any agent logic runs, the workflow has to work out what it just received and turn it into something uniform. Images and PDFs go down different paths — a PDF needs parsing before anything can look at it. Voice notes need transcription. Only after that does the agent see a consistent picture of what the user actually asked for.
Budget real time for this. It is unglamorous branching and it is most of the build.
Conversations are not requests
The second thing that catches people out is state. A web form is one request and one response. A WhatsApp thread is a conversation that might run over an hour, with the user wandering off and coming back.
"Try it in a warmer palette" only means something if the system still knows which room, which floor plan, and which of the four previous versions is being referred to. That means conversation state keyed to the person, holding not just message history but the artifacts already produced. Without it, every message is treated as a fresh brief and the user has to re-explain themselves — which, in a chat interface, feels broken in a way it would not on a web form.
Design for the medium, not around it
The temptation is to fight WhatsApp: force structure, ask for a form, restrict input types. It is the wrong instinct. The reason a client wants this is that their customers already live in that app and will not adopt anything else. Every constraint you add gives back the advantage you were building for.
Better to accept whatever arrives and do the normalising work behind the scenes. The person sending a voice note while walking around a room should not have to know that a transcription step exists.
What you need in place
- WhatsApp Business Cloud API access — this is a business platform integration, not a personal WhatsApp account.
- A branch for every input type you will accept: text, image, document, audio. Decide up front which ones you will not support, and reply helpfully when they arrive anyway.
- Transcription for voice notes, and document parsing for anything that arrives as a PDF.
- Conversation state keyed to the sender, holding both history and any artifacts already generated.
- A plan for long-running work. Generating images takes longer than a chat reply, so tell the user something is happening rather than leaving silence.
Done properly it stops feeling like a bot and starts feeling like messaging someone who is good at their job. That is the whole point, and it is mostly plumbing.