Top 5 Features of LlamaIndex

1.  Data Loading

Creating LLM applications that can act on your data requires processing and loading that data. LlamaIndex achieves this through data connectors, also known as Reader. These connectors can ingest data from various sources and format it into Document objects, which are collections of text data and metadata.

LlamaIndex's SimpleDirectoryReader is a straightforward class that creates documents from every file in a specified directory. It can read from a variety of file types, including Markdown, PDFs, Word documents, PowerPoint decks, images, audio, and video.

LlamaHub is a registry provided by LlamaIndex where users can download data connectors for their specific use cases.

LlamaParse is a state-of-the-art tool for loading PDF files, available as a managed API. It is designed to create RAG systems over complex PDFs containing embedded tables and charts. LlamaIndex offers a proprietary parsing service that excels at converting PDFs with complex tables into well-structured markdown formats. This representation can be further parsed using advanced Markdown parsing and recursive retrieval algorithms available in the open-source library. LlamaParse enables users to build RAG systems over complex documents that can answer questions involving both tabular and unstructured data.

Additionally, LlamaIndex offers various methods for transforming loaded data, such as node parsers and text splitters, making it easier for LLMs to process the data.

2. Data Indexing

To quickly fetch relevant information from a user query, we need to index our data. An index is a data structure that is a core component for RAG use cases. Indexing is the step that follows the loading and transformation of data. Indexes store data in Node objects, which are created after splitting the original documents into chunks. There are various types of indexes:

• Summary Index: Stores nodes as a sequential chain. Retrieved nodes can be filtered with, for example, a keyword filter or a top-k neighbor parameter, and are then loaded into the Response Synthesis module that generates a response.

• Vector Store Index: Stores each node and its corresponding embedding in a Vector Store. When querying, the top-k most similar nodes are passed into the Response Synthesis module.

• Tree Index: Represents a hierarchical structure of nodes. Querying a tree index involves traversing from root nodes down to leaf nodes. The child_branch_factor can be set to decide how many child nodes should be selected for a given parent node. The selected leaf nodes are then passed into the Response Synthesis module.

• Keyword Table Index: Maps keywords found in the nodes to the corresponding nodes. When querying, relevant keywords are extracted from the query and matched with the keywords in the table index. The nodes that map to these keywords are passed into the Response Synthesis module.

3. Engines

Indexed data can be queried using the Query and Chat Engines. The Query Engine is a generic interface that allows users to ask questions about their data. The Chat Engine is an interface for having a conversation with your data.

Query Pipelines

At the beginning of 2024, LlamaIndex introduced Query Pipelines, an API within LlamaIndex that allows users to orchestrate simple-to-advanced query workflows over their data.

QueryPipeline

QueryPipeline is an abstraction that can integrate various LlamaIndex modules, such as LLMs, prompts, query engines, and retrievers. It can create a computational graph over these modules, forming either a Sequential Chain or a Directed Acyclic Graph (DAG). Retrieval Augmented Generation (RAG) systems involve a lot of query orchestration. Building an advanced RAG pipeline, consisting of query transformations, multi-stage retrieval algorithms, and the use of prompts and LLMs optimized for performance, can be a complex task.

The advantages of QueryPipeline are:

• Creating query workflows with less code: The outputs of some modules do not have to be manually converted into the inputs of other modules. This means that QueryPipeline handles the exact typing arguments for each module.
  
  
• Better readability: Reduced complexity results in better readability.
  
  
• End-to-end observability: Integrate callback functions across the entire pipeline.

Sequential Chains

Sequential Chains are simple pipelines where the output of the previous module directly goes into the input of the next module. Some examples:

• Prompt -> LLM -> Output parsing
• Retriever -> Response synthesizer

DAG for Advanced RAG Workflow

To build an advanced RAG workflow, consisting of rewriting, retrieval, reranking, synthesis, etc., users need to define the modules they want to use. Furthermore, they need to define the query pipeline and integrate the defined modules. Finally, the edges between the modules must be added to build the relationships between the modules.

4. Evaluation and Observability

Evaluation and benchmarking are fundamental to the development of LLM applications. In this context, LlamaIndex provides essential tools for assessing and enhancing the performance of LLM applications, particularly RAG systems. The two main components of evaluation are Response Evaluation and Retrieval Evaluation.

Response Evaluation

Assessing the quality of generated responses presents unique challenges due to the nuanced nature of language understanding. LlamaIndex addresses this with LLM-based evaluation modules that measure the quality of results against various criteria, including:

• Correctness: Does the generated response align with the reference answer given the query?
  
  
• Semantic Similarity: How closely does the predicted answer resemble the reference answer in meaning?
  
  
• Faithfulness: Is the generated answer faithful to the retrieved contexts, avoiding the introduction of false information?
  
  
• Context and Answer Relevancy: Are the retrieved context and generated answer relevant to the query?
  
  
• Guideline Adherence: Does the generated response adhere to predefined guidelines or standards?

Question Generation

Beyond response evaluation, LlamaIndex facilitates question generation using the provided data. This feature enables the automatic creation of questions for evaluation purposes, allowing developers to assess the LLM's accuracy in answering queries based on the available dataset.

Retrieval Evaluation

LlamaIndex also offers modules for independent retrieval evaluation. By leveraging established ranking metrics like mean-reciprocal rank (MRR), hit rate, and precision, these modules enable a comprehensive assessment of retriever performance. The retrieval evaluation process includes two core steps:

• Dataset Generation: Synthetic creation of (question, context) pairs from unstructured text corpora.
  
  
• Retrieval Evaluation: Assessment of retrieved results using ranking metrics to evaluate the retriever against a given set of questions.

Instrumentation

The new instrumentation module fundamentally transforms how developers can track events within LlamaIndex applications. It includes several key classes:

• Event: Represents a singular moment in the execution of the application, capturing specific occurrences.
  
  
• EventHandler: Listens to events and executes custom logic in response to these occurrences.
  
  
• Span: Represents the execution flow within the application, encapsulating multiple events.
  
  
• SpanHandler: Manages the lifecycle of spans, including their entry, exit, and potential early termination due to errors.
  
  
• Dispatcher: Acts as the central hub, transmitting events and span-related signals to the appropriate handlers.

LlamaIndex's instrumentation module empowers developers to gain deeper insights into their applications' behavior by effectively tracking events and spans. By leveraging this powerful tool, developers can enhance application monitoring, debugging, and performance optimization, ultimately leading to more robust and efficient LLM applications.

5. llama-index-networks

llama-index-networks is a llama-index library extension that enables the creation of a network of RAG systems built on external data sources and supplied by external actors. This network paradigm facilitates context augmentation, allowing for collaboration across various data sources and actors. Users can easily access these RAG systems with just a few lines of code.

Specifically, actors with their own set of documents and a RAG system must build a ContributorService around their QueryEngine and expose it via the standard llama-index Network Contributor Service.

Actors who wish to connect to the RAG systems in the llama-index Network Contributor Service can build a NetworkQueryEngine to connect to a list of ContributorServices.