Case Study | Building an AI-Powered City Information and Extraction Platform
Building an AI-Powered City Information and Extraction Platform
Generative AIMulti-Agent SystemsRAG
FiftyFive built an LLM-powered web extraction platform and German-language AI assistant for the client. We developed multi-agent extraction, multi-site crawling, semantic search, RAG, date-aware query routing, and source-linked responses to make information across multiple public websites easier to retrieve and use.
The client needed a single conversational entry point to information about the client. Residents and visitors look for answers across the city portal, the tourism site, the city library, the regional transport network (naldo) and federal services such as residence registration, pension services and certificates of conduct. Each site has its own layout, many pages are dynamic, and much of the useful detail sits inside PDFs. The client first needed a dependable way to extract this information, and then an assistant that answers in natural German, links to its sources and handles time-sensitive questions such as upcoming events.
The Need
Challenges
Turning many differently built websites into reliable answers meant solving extraction, retrieval quality and date accuracy together. A wrong opening time, or a past event shown as upcoming, would damage trust in an official city channel.
Rule-based scrapers break whenever a site changes its layout, and every new site needs its own rules. Extraction had to understand what a page says rather than depend on fixed selectors.
Many pages are rendered with JavaScript or written in different languages. The platform had to handle dynamic and multilingual pages with the same pipeline as simple static ones.
Information was spread across several independent domains with different sitemaps and crawl rules, and much of it was in PDFs. The platform had to find every relevant page and document, respect robots.txt and avoid duplicate URLs.
Content was in German and full of administrative vocabulary. Retrieval had to work on meaning, so a question in everyday language still reached the right official page.
Some questions need no lookup, while others are too vague to answer. Sending every message through retrieval would add latency and noise, and answering vague questions directly would produce guesses.
Event questions depend on time. Phrases such as “tomorrow” or “this weekend” had to be resolved to real dates, with only events in that window returned.
Vector search returns many loosely related passages, so a second ranking stage was needed to give the language model only the most relevant content, with its sources.
Our Approach
Solution
FiftyFive delivered the platform in two phases. Phase one replaced rule-based scraping with a multi-agent LLM extraction pipeline that reads pages the way a person would and returns structured data. Phase two used that crawling foundation to index about 9,200 URLs across five public domains into a multilingual vector database. A FastAPI assistant then routes each question, resolves dates, retrieves and reranks content, and answers in German with events and source links.
Phase 1: LLM-powered web extraction platform
FiftyFive built a general-purpose extraction platform that works on new websites without writing site-specific rules.
Multi-agent LLM orchestration in which agents analyse a page, understand its context and extract structured information.
Google Gemini zero-shot semantic understanding in place of site-specific extraction rules.
Playwright browser automation for dynamic, JavaScript-rendered pages.
DOM parsing with BeautifulSoup to isolate the main content before extraction.
Structured output for event and product details.
Multilingual support so the same pipeline works on websites worldwide.
Phase 2: Multi-site crawler and document ingestion
FiftyFive built a crawler that discovers content on each official domain and feeds it into the index.
Sitemap-first discovery, including nested sitemap indexes, with a multi-threaded fallback crawl when no sitemap exists.
robots.txt compliance and polite request delays on every domain.
URL normalisation that strips tracking parameters and fragments to remove duplicate pages.
Text extraction from linked PDF documents with pdfplumber.
A re-indexing API endpoint that refreshes content for any list of sites on demand.
Phase 2: Multilingual semantic index
FiftyFive designed the index to retrieve German content by meaning, with every passage traceable to its source.
Sentence-aware chunking at 512 tokens with overlap, so passages keep their context.
BAAI bge-m3 multilingual embeddings (1,024 dimensions) built with LlamaIndex.
Pinecone serverless vector storage with cosine similarity.
Source URL stored with every chunk for attribution in answers.
Phase 2: Query routing and date understanding
FiftyFive added a decision layer in front of retrieval so the assistant responds appropriately to each kind of question.
A Gemini 2.0 Flash routing step that answers simple questions directly, asks a clarifying question when a request is unclear, or triggers retrieval.
Relative date resolution with dateparser, converting phrases like “morgen” into exact dates before search.
Event-aware retrieval that widens the result set for event questions.
Conversation history carried across turns for follow-up questions.
Phase 2: Two-stage retrieval and grounded answers
FiftyFive built the answer pipeline to keep responses accurate, current and linked to official sources.
Top-20 vector retrieval followed by reranking with the bge-reranker-v2-m3 model.
The top 7 passages (15 for event queries) passed to the language model as context.
Answers grounded only in retrieved content, written in natural German, with a polite fallback when information is unavailable.
Structured JSON output with the answer, events (name, time, location, link) and source links.
Technology
Technologies
Language & API
PythonFastAPIUvicorn
LLM
Google Gemini (zero-shot extraction)Gemini 2.0 Flash (assistant)
Agent architecture
Multi-agent LLM orchestrationPrompt engineering
RAG framework
LlamaIndex
Embeddings & reranking
BAAI bge-m3 (Hugging Face)bge-reranker-v2-m3
Vector database
Pinecone (serverless)
Crawling & extraction
PlaywrightBeautifulSoupRequestspdfplumberrobots.txt and sitemap parsing
Language processing
dateparsertiktoken
Deployment
Docker
Experts
Team
2
AI Engineers
The Impact
Results
FiftyFive delivered an extraction platform that adapts to new websites without custom rules, and a German-language assistant built on top of it. Residents and visitors get answers about the client with links to the original pages, instead of searching several websites.
Extraction without site-specific rules
LLM agents with zero-shot understanding pull structured event and product data from static, dynamic and multilingual websites.
One entry point to municipal information
About 9,200 URLs across five public domains, including linked PDFs, are searchable through one chat interface and can be re-indexed on demand.
Grounded, source-linked answers
Two-stage retrieval and reranking keep answers tied to official content, and every answer links to its sources.
Time-aware event answers
Relative dates are resolved before search, so event questions return only events in the requested window.
Support
FAQs
It gives residents and visitors one German-language conversational entry point for information drawn from the city portal, tourism site, library, regional transport network and federal services. Answers link back to the original sources.
Multi-agent LLM extraction uses Google Gemini to interpret page content without site-specific selector rules. Playwright renders dynamic pages, and BeautifulSoup isolates main content before structured extraction.
About 9,200 URLs across five public domains are indexed, including text extracted from linked PDF documents.
The crawler starts with sitemaps, follows nested sitemap indexes and uses a fallback crawl where no sitemap exists. It respects robots.txt, adds polite delays and normalises URLs to remove duplicates.
The platform splits content into overlapping 512-token chunks and creates multilingual BAAI bge-m3 embeddings with LlamaIndex. Pinecone stores the vectors for similarity search, with a source URL on each chunk.
The system retrieves the top 20 vector matches and reranks them with bge-reranker-v2-m3. The assistant receives the top 7 passages, or 15 for event questions.
A Gemini 2.0 Flash routing step can ask for clarification instead of guessing. It answers simple questions directly and sends questions that need official content through retrieval.
dateparser resolves relative expressions such as “morgen” into exact dates before searching. Event-aware retrieval then uses the requested time window.
Each indexed passage stores its source URL. The assistant is instructed to answer from retrieved content and returns structured JSON with the answer, any event details and source links.
Yes. Playwright renders JavaScript-driven pages, while pdfplumber extracts text from linked PDF documents for indexing.
A re-indexing API endpoint refreshes content for any specified list of sites on demand.
Two AI Engineers built the extraction platform and German-language assistant for a government and public-sector client in Germany.
We use cookies to personalise content and ads, to provide social media
features and to analyse our traffic. We also share information about
your use of our site with our social media, advertising and analytics
partners who may combine it with other information that you’ve provided
to them or that they’ve collected from your use of their services.
Necessary cookies help make the website usable. Analytics cookies help
us understand how visitors interact with the website. Marketing
cookies may be used to deliver relevant ads and measure campaign
performance.
Necessary cookies
Required for core website functionality. These cannot be
disabled.
Analytics cookies
Helps us measure traffic and improve website performance.
Marketing cookies
Used for advertising, remarketing, and campaign tracking.
