llm app

Unlike standard RAG applications where documents are indexed once and queries are served from a static index, LLM App maintains live connections to data sourcesspreadsheets, databases, document stores, APIsand incrementally updates the knowledge index as the underlying data changes, ensuring answers reflect the current state of the data rather than a stale snapshot.

The framework handles the full pipeline from live data ingestion through embedding, indexing, and query serving: data connectors poll or subscribe to source changes, the changed chunks are re-embedded and their index entries updated, and the query interface always serves from the current index state.

This architecture eliminates the operational burden of manually re-running indexing pipelines when source data updatesa significant friction point for production RAG applications where the underlying data changes frequently.

Teams building enterprise knowledge bases over frequently-changing internal documentation, financial applications serving questions over live market data or analyst reports, and developers constructing assistants over operational databases where queries must reflect current system state use LLM App to achieve the freshness guarantees that static indexing pipelines cannot provide.

The framework is particularly valuable for use cases where data staleness has meaningful business consequencescustomer support agents serving over live product documentation, or compliance tools querying current regulatory guidance that updates regularly.