Introduction

Imagine having a personal assistant that not only understands your requests but also knows exactly how to execute them, whether it’s a quick calculation or fetching the latest stock market news. In this article, we’ll dive into the fascinating world of AI agents and explore how to build your own using the LlamaIndex framework. We’ll walk you step-by-step through the process of creating these intelligent agents, highlighting the power of LLM’s function calling capabilities and demonstrating how they can make decisions and perform tasks with impressive efficiency. Whether you’re new to AI or an experienced developer, this guide will show you how to unlock the full potential of AI agents in just a few lines of code.

Learning outcomes

• Understand the basics of AI agents and their problem-solving capabilities.
• Learn how to implement AI agents using the LlamaIndex framework.
• Explore the function calling features in LLMs for efficient task execution.
• Learn how to integrate web search tools into your AI agents.
• Gain hands-on experience building and customizing AI agents with Python.

This article was published as part of the Data Science Blogathon.

What are AI agents?

AI agents are like digital assistants on steroids. They don’t just respond to your commands, they understand, analyze, and make decisions about the best way to execute those commands. Whether it’s answering questions, performing calculations, or retrieving the latest news, AI agents are designed to perform complex tasks with minimal human intervention. These agents can process natural language queries, identify key details, and use their skills to provide the most accurate and useful answers.

Why use AI agents?

The rise of AI agents is changing the way we interact with technology. They can automate repetitive tasks, improve decision-making, and deliver personalized experiences, making them invaluable across industries. Whether you work in finance, healthcare, or e-commerce, AI agents can streamline operations, improve customer service, and provide deep insights by performing tasks that would otherwise require a lot of manual effort.

What is LlamaIndex?

LlamaIndex is an advanced framework designed to simplify the process of building AI agents using Large Language Models (LLMs). It leverages the power of LLMs such as OpenAI’s models, allowing developers to create intelligent agents with minimal coding. LlamaIndex allows you to plug in custom Python functions and the framework will automatically integrate them with the LLM, allowing your AI agent to perform a wide range of tasks.

Key Features of LlamaIndex

• Function call: LlamaIndex enables AI agents to invoke specific functions based on user queries. This feature is essential for creating agents that can handle multiple tasks.
• Tool integration:The framework supports the integration of various tools including web search, data analysis, and more, allowing your agent to perform complex operations.
• Ease of use: LlamaIndex is designed with a user-friendly design, making it accessible to both beginners and experienced developers.
• Adaptability: With support for custom functions and advanced features such as pydantic models, LlamaIndex provides the flexibility needed for specialized applications.

Steps to Deploy AI Agents Using LlamaIndex

Now let’s look at the steps for implementing AI agents using LlamaIndex.

Here we use OpenAI’s GPT-4o as our LLM model, and the web querying is performed using Bing search. Llama Index already has Bing search engine integration, and can be installed with this command.

!pip install llama-index-tools-bing-search

Step 1: Get the API key

First, you need to create a Bing Search API key, which you can get by creating a Bing resource using the link below. For experimentation, Bing also offers a free tier with 3 calls per second and 1k calls per month.

Step 2: Install the required libraries

Install the required Python libraries using the following commands:

%%capture

!pip install llama_index llama-index-core llama-index-llms-openai
!pip install llama-index-tools-bing-search

Step 3: Set the environment variables

Next, set your API keys as environment variables so that LlamaIndex can access them at runtime.

import os

os.environ("OPENAI_API_KEY") = "sk-proj-"
os.environ('BING_API_KEY') = ""

Step 4: Initialize the LLM

Initialize the LLM model (in this case OpenAI’s GPT-4o) and run a simple test to confirm it works.

from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-4o")
llm.complete("1+1=")

Step 5: Create two different functions

Create two functions that your AI agent will use. The first function performs simple addition, while the second retrieves the latest stock market news using Bing Search.

from llama_index.tools.bing_search import BingSearchToolSpec


def addition_tool(a:int, b:int) -> int:
    """Returns sum of inputs"""
    return a + b
    

def web_search_tool(query:str) -> str:
  """A web query tool to retrieve latest stock news"""
  bing_tool = BingSearchToolSpec(api_key=os.getenv('BING_API_KEY'))
  response = bing_tool.bing_news_search(query=query)
  return response

For a better function definition we can also use pydantic models. But for simplicity we rely here on the ability of LLM to extract arguments from the user query.

Step 6: Create a function tool object from user-defined functions

from llama_index.core.tools import FunctionTool


add_tool = FunctionTool.from_defaults(fn=addition_tool)
search_tool = FunctionTool.from_defaults(fn=web_search_tool)

A function tool allows users to easily convert any user-defined function into a tool object.

In this case, the function name is the tool name and the doc string is considered the description. However, you can override this as below.

tool = FunctionTool.from_defaults(addition_tool, name="...", description="...")

Step 7: Call the predict_and_call method with the user’s query

query = "what is the current market price of apple"

response = llm.predict_and_call(
    tools=(add_tool, search_tool),
    user_msg=query, verbose = True
)

Here we call the predict_and_call method of llm, together with the user query and the tools we defined above. Tools arguments can contain more than one function by putting all the functions in a list. The method iterates through the user query and decides which tool is best suited to perform the given task from the list of tools.

Sample output

=== Calling Function ===
Calling function: web_search_tool with args: {"query": "current market price of Apple stock"}
=== Function Output ===
(('Warren Buffett Just Sold a Huge Chunk of Apple Stock. Should You Do the Same?', ..........

Step 8: Putting it all together

from llama_index.llms.openai import OpenAI
from llama_index.tools.bing_search import BingSearchToolSpec
from llama_index.core.tools import FunctionTool

llm = OpenAI(model="gpt-4o")

def addition_tool(a:int, b:int)->int:
    """Returns sum of inputs"""
    return a + b
    

def web_search_tool(query:str) -> str:
  """A web query tool to retrieve latest stock news"""
  bing_tool = BingSearchToolSpec(api_key=os.getenv('BING_API_KEY'))
  response = bing_tool.bing_news_search(query=query)
  return response
 

add_tool = FunctionTool.from_defaults(fn=addition_tool)
search_tool = FunctionTool.from_defaults(fn=web_search_tool)

query = "what is the current market price of apple"

response = llm.predict_and_call(
    tools=(add_tool, search_tool),
    user_msg=query, verbose = True
)

Advanced Customization

For those looking to push the boundaries of what AI agents can do, advanced customization provides the tools and techniques to refine and extend their capabilities, allowing your agent to perform more complex tasks and deliver even more accurate results.

Improved function definitions

To improve how your AI agent interprets and uses features, you can include pydantic models. This adds type checking and validation so your agent processes input correctly.

Dealing with complex questions

For more complex user queries, you may consider creating additional tools or refining existing tools to handle multiple tasks or more complex requests. This may mean adding error handling, logging, or even custom logic to manage how the agent responds to different scenarios.

Conclusion

AI agents can process user input, reason about the best approach, consult relevant knowledge, and perform actions to provide accurate and useful answers. They can extract parameters specified in the user’s query and pass them to the relevant function to perform the task. With LLM frameworks like LlamaIndex, Langchain, etc., one can easily implement agents with a few lines of code and also customize things like function definitions using pydantic models.

Key Points

• Agents can perform multiple independent functions and determine which function to perform based on the user’s query.
• Using Function Calling, LLM determines which function is best to perform the task, based on the function name and description.
• The function name and description can be overridden by explicitly specifying the function name and description parameter when creating the tool object.
• Llamaindex has built-in tools and techniques to implement AI agents in a few lines of code.
• It is also important to note that function calling agents can only be implemented using LLMs that support function calls.

Frequently Asked Questions

The media shown in this article is not owned by Analytics Vidhya and is used at the author’s sole discretion.