How to Build an AI Chatbot: Lessons from Five Cultural Chatbots by DEN

1. Start with a Clear Goal

A good chatbot doesn’t start with technology, but with a clear goal. What should the bot be capable of? Who are you building it for? And when is it ‘good enough’?

Chatbots with a defined task work better than bots that try to do everything at once. Think: “assist with drafting a marketing plan” instead of “do the marketing”. The more specific the task, the better the result. Determine when the result is ‘good enough’: for example, whether the tone is correct or if the answers are more consistent than those from a casual chat.

During the development of the cultural chatbots, the five tasks were quickly identified: a brainstorming session with the team provided enough input for more than five chatbots. However, specifying the task proved to be a challenge. The chatbots needed to provide more culture-specific answers than, for instance, ChatGPT. At the same time, the chatbots had to be broadly applicable: for various tasks and different sub-sectors.

Before starting, determine:

1. One concrete goal or task
2. Who the user is: your audience, a specific colleague, or all your colleagues?
3. When the chatbot is 'good enough' to use

2. Set Selection Criteria

When building your own chatbot, you can make various design choices. The key considerations are:

• Do you choose one provider, or do you want the option to switch providers in the future? Choosing one provider makes you dependent on them, even if they raise prices or competitors improve. What sustainable choices are you making? A smaller language model (e.g., ChatGPT’s 4o-mini or Claude’s Haiku 4.5) has less environmental impact than a large model (e.g., ChatGPT’s 5 or Claude’s Opus 4.6), but the responses are less extensive.

• Do you want insight into the usage of your chatbot? If so, to what extent: number of users or also the content of the prompts? Usage insights help improve the chatbot but may be perceived as a privacy violation.

• Can the chatbot use information from your internal database in addition to the knowledge base? Responses can be more complete if your internal database is used, but this can also ‘muddy’ the answers.

For the development of the cultural chatbots, free choice of provider and language model were very important. Currently, ChatGPT is being used, but a switch to Claude is being considered. Additionally, we prioritized user privacy over usage insights. We chose a relatively small language model as it met our requirements and has less environmental impact. These choices come with pros and cons, which we’ll discuss later.

3. Build a Knowledge Base

A chatbot is only as good as the information you put into it. But beware: collecting, cleaning, and structuring documents often takes more time than expected. What information is really necessary or relevant to perform the task well?

Also consider: what information are you actually allowed to use? Policy documents, project reports, or manuals can be valuable but be mindful of privacy and security risks.

At DEN, we started with the materials we already had, both our own and from our internal knowledge base. Since this was a time-consuming task, the cultural chatbots were launched with a relatively small knowledge base, which was gradually expanded. For example, we included basic information about all cultural funds supporting digital projects.

4. Give Your Chatbot Direction 

Chatbots operate based on ‘prompts’ and system instructions. Think of this as the compass for your bot: it determines the tone, tasks, and boundaries of the responses.

A well-formulated instruction can make the difference between a useful and vague response. An instruction like "you are an experienced fundraiser who assists with writing cultural grant applications" provides much more direction than "help with grants."

You can also guide the language used: do you want an informal tone or a formal one? Do you use conversational language or policy jargon? Should the chatbot respond to irrelevant questions as well?

We have chosen to make the prompts we used public, so everyone can use them to create chatbots that are even better tailored to their own situation.

5. Test with Users

Don’t build your chatbot in one go; work towards the final product in several iterations. You’ll likely want to test the first prototypes yourself. For subsequent versions, it’s essential to involve actual users, whether they’re colleagues or your audience. Their feedback helps improve language, structure, and output. Do this in multiple rounds.

From user testing, we learned that users initially found the responses too long, while we thought ‘more explanation’ was better. They also asked for more concrete examples instead of general advice, prompting us to enrich our knowledge base.

6. Ensure Maintenance and Development

A chatbot is never ‘finished.’ Information changes, new questions arise, and technology evolves rapidly. Without maintenance, your chatbot will quickly become outdated. Also, think about scalability. You may want to expand to other tasks or departments later. Make sure you have a solid foundation.

We underestimated the maintenance and development of the cultural chatbots. Keeping the knowledge base up to date is a continuous task. Although prioritizing user privacy was a deliberate choice, we lack user insights to further develop the cultural chatbots. We don’t know exactly how and for what purposes the chatbots are being used.

Ready to Get Started?

An AI chatbot can be a valuable tool for your work or your audience. But remember, it needs to be well-instructed, guided, and continuously trained. Read this article to learn how to use generative AI to perform tasks for you.  (opens in new tab)