Kognita
All posts
Tutorial2026-10-17·8 min read

Building a Slack bot powered by your knowledge base

The best place to answer a question is where it's already being asked — and in most companies, that's Slack. Here's how to build an /ask command that answers from your knowledge base, with sources.

People don't visit knowledge portals. They ask in Slack. That's not a failure of discipline — it's just where the conversation already is, and asking a channel is faster than remembering a separate tool exists. So the highest-adoption place to put your knowledge base isn't a portal at all; it's a Slack bot that answers questions in the channel where they're asked. Here's how to build an /ask command that retrieves from your knowledge base and answers with citations — meeting people exactly where they already are.

building a Slack bot powered by your knowledge base

Why Slack is the right surface

Adoption is the metric that decides whether a knowledge base delivers value, and adoption follows friction. A portal people must remember to open, in a tab they don't have, loses to "just ask in Slack" every time. A Slack bot flips that: the knowledge base becomes reachable inside the tool people already live in, with zero context-switch. The question they'd have asked a colleague, they ask the bot — same channel, same flow, but the answer is instant, consistent, and sourced. You're not asking anyone to change their behavior; you're inserting a better answer into the behavior they already have.

The architecture

The bot is a thin layer over the same retrieve-then-generate loop behind any grounded assistant:

  1. A user invokes a slash command (/ask <question>) or mentions the bot.
  2. Slack sends the event to your backend.
  3. Your backend runs hybrid search over the knowledge base for the question.
  4. It builds a grounded prompt and calls the model to generate an answer with citations.
  5. It posts the answer back to the channel, formatted with sources.

The Slack-specific work is steps 1–2 and 5 (receiving events and formatting replies); the RAG core (3–4) is identical to a web bot.

Step 1: Receive the slash command

Register a slash command (or an app mention) in your Slack app config, pointed at your backend endpoint. Slack posts the command there. Two Slack realities shape the handler:

  • You must respond within 3 seconds, but retrieval + generation takes longer. So acknowledge immediately and post the real answer asynchronously.
  • Verify the request is really from Slack using the signing secret, exactly as you'd verify any inbound webhook.
def handle_slash_command(request):
    if not verify_slack_signature(request):
        return 401
    payload = parse(request)
    enqueue_answer_job(payload["text"], payload["channel_id"], payload["response_url"])
    return { "text": "Searching the knowledge base…" }   # instant ack

Step 2: Retrieve and generate

A worker picks up the job and runs the familiar loop — hybrid search, then grounded generation:

def build_answer(question):
    passages = hybrid_search(question, limit=5)   # KB search API
    context = "\n\n".join(f"[{p['source']}]\n{p['content']}" for p in passages)
    answer = llm(
        system="Answer only from the sources. Cite each claim. If the sources "
               "don't cover it, say so and suggest asking the team.",
        user=f"Sources:\n{context}\n\nQuestion: {question}",
    )
    return answer, passages

Hybrid search earns its keep in Slack especially — people ask about exact product names, error codes, and internal jargon that pure vector search fumbles, mixed with natural-language questions. Fusing keyword and semantic retrieval handles both.

Step 3: Post the answer with sources

Send the result back via the response_url Slack gave you, formatted with Slack's block kit so the answer is readable and the sources are visible:

def post_answer(response_url, answer, passages):
    sources = "\n".join(f"• <{p['url']}|{p['source']}>" for p in passages)
    requests.post(response_url, json={
        "blocks": [
            { "type": "section", "text": { "type": "mrkdwn", "text": answer } },
            { "type": "context", "elements": [
                { "type": "mrkdwn", "text": f"*Sources:*\n{sources}" } ] },
        ]
    })

Showing sources matters as much in Slack as anywhere: it lets people verify, builds trust, and turns the bot's answer into a jumping-off point to the real docs.

Details that make it good

  • Post publicly, thoughtfully. Answering in-channel (rather than ephemerally) means the next person with the same question sees the answer already there — compounding value. But respect channel norms; some teams prefer the bot reply in a thread.
  • Respect permissions. If your knowledge base spans content with different access levels, scope retrieval to what the asking user is allowed to see. A Slack bot that surfaces restricted content to the wrong person is a leak, not a feature. Filter by the user's permitted scope.
  • Escalate gracefully. When retrieval finds nothing, the bot should say so and point to a human or channel — "I don't have that in the knowledge base; try #team-help." An honest miss beats a confident guess, and it keeps trust intact.
  • Capture feedback. Add reaction-based or button feedback (👍/👎) so you can measure answer quality and spot coverage gaps — the questions the bot fails are your documentation backlog.

The bigger payoff

A knowledge-base Slack bot does something beyond answering questions: it closes the loop on documentation debt. Every repeated question in Slack was a signal of a knowledge gap; now those same questions get answered from the KB automatically, and the ones the bot can't answer are surfaced as exactly the content you're missing. The bot both serves the existing knowledge and maps the holes in it.

And because it lives where people already ask, it actually gets used — which is the thing every knowledge base project is really fighting for. The fanciest retrieval in the world delivers zero value in a portal nobody opens. The same retrieval behind /ask in the channel people already type in gets used all day. Meet the questions where they're asked, answer with sources, escalate honestly when you can't — that's a Slack bot that turns your knowledge base into something the team actually reaches for.