Thank you for coming to this blog series. In my first blog, I introduced the idea of Foundation Models for SAP and how to deploy them on our private account, private vs. public access topics, which will evolve.
In my second blog, I introduced the concept of Model Customizing, meaning that for an existing Foundation Model, how to infuse it with a new dataset without the need to pre-train it; this is important if you like a conversational AI in particular, but we want to feed it with our Enterprise data, and providing a way to do it called RAG aka Retrieval Augmented Generation, a relatively new methodology for Models to retrieve facts from an external knowledge base on most accurate, up-to-date information.
This is possible with Natural Language Queries (NLQ) and LangChain Agents.
LangChain provides two main methods to interact with SQL Databases: using Chains for Query creation and execution. On top of the Chain, another technique is to optionally interact with SQL databases using Agents for more flexible querying. LangChain SQL Agent provides a more flexible way of interacting with SQL Databases than the
What does all this mean?
🧑🏻💻 We ask a question without a query, "How many sales did we have this week for this company code?"
❓ LLM doesn't know about queries, goes to LangChain SQL Agent / Chain, it translates customer question into an SQL Query
🛢 DB is queried using the SQLAlchemy library (or others)
🤖 LLM now has all the information it needs and can provide an answer to the user
The way that Langchain works is that you can configure it as a library inside of your application and connect it to different endpoints.
In our example, we connected to a database and an API endpoint. Still, we've also got a large language model, which gives us our natural language capability to query the database using natural language.
We might ask the LLM, "How many Materials did we create yesterday?"
The application receives the prompt "How many Materials did we create yesterday?" and sends that through LangChain.
LLMs can't execute SQL Queries; LLMs are text encoders or decoders, not SQL executors. Here is where we interact with LangChain Agents. Agents will receive the SQL query generated and will execute the SQL query against the database to get the information.
The whole thing is about prompt engineering; we did in-context learning. I just introduced how it would work with an SQL query, but this can be applied to many other topics; we can use Agents to execute Matemathic calculations (LLMs don't do mathematics; they are text encoders and decoders), they can book a flight for us if they can connect to any SAP API from any other application, which I will describe in other blogs.
While Langchain is a framework for Generative AI application development and orchestration, LlamaIndex (formerly GPT Index) is a data framework for LLM applications to ingest, structure, and access private or domain-specific data.
LlamaIndex vectorized the data, as we described in previous blogs, to prepare the data into a format that can be understood by the LLM. The process primarily encompasses two key phases: the indexing phase and the querying phase. The primary purpose of using LlamaIndex is the standardization of different sources and the performance increase this will provide.
In my second blog, I introduced the concept of Model Customizing, meaning that for an existing Foundation Model, how to infuse it with a new dataset without the need to pre-train it; this is important if you like a conversational AI in particular, but we want to feed it with our Enterprise data, and providing a way to do it called RAG aka Retrieval Augmented Generation, a relatively new methodology for Models to retrieve facts from an external knowledge base on most accurate, up-to-date information.
| In this third blog, I am not leaving the same concept, leveraging existing LLMs and (for the moment) avoiding the pre-training topic. I firmly believe that enterprises will have their own LLMs using one of the existing models as a base. That's out of discussion, but the evolution of the LLMs is so intense that it will probably take some time to get to that point. Pre-training models is not an easy thing and is not cheap either. In this series of blogs, I am introducing a way to interact with existing LLMs with techniques that do not require pre-training the models on our own datasets. Then, in this blog, I present what I believe is an exciting topic: While building an end-to-end solution, where a conversation will end by accessing live data from another application. |
This is possible with Natural Language Queries (NLQ) and LangChain Agents.
Natural language querying allows users to interact with databases; leveraging the power of LangChain, SQL Agents, and Large Language Models (LLMs), we can create applications that enable users to query databases using natural language.
LangChain
LangChain is a framework designed for building applications powered by language models. It provides a standard chain interface, integrates with various tools, and offers end-to-end application chains. The two main features of LangChain are data-awareness and agentic behavior.
Data awareness enables the language model to connect to other data sources, while agentic behavior allows it to interact with its environment. Using agents, LangChain can dynamically decide which tools to call based on user input. This makes agents extremely powerful when used correctly.
LangChain provides two main methods to interact with SQL Databases: using Chains for Query creation and execution. On top of the Chain, another technique is to optionally interact with SQL databases using Agents for more flexible querying. LangChain SQL Agent provides a more flexible way of interacting with SQL Databases than the
SQLDatabaseChain.What does all this mean?
🧑🏻💻 We ask a question without a query, "How many sales did we have this week for this company code?"
❓ LLM doesn't know about queries, goes to LangChain SQL Agent / Chain, it translates customer question into an SQL Query
🛢 DB is queried using the SQLAlchemy library (or others)
🤖 LLM now has all the information it needs and can provide an answer to the user
Why use Agents on top of Chains The main advantages of using the SQL Agent are:
|
This blog uses Langchain to connect the application with LLM and External Data Sources, such as an Oracle DB for querying.
How Agents work
The SQL Database Agent from LangChain is designed to interact with any database, allowing users to ask questions in natural language and receive answers.
from langchain import Cohere, SQLDatabase, SQLDatabaseChain
from langchain.agents.agent_toolkits import SQLDatabaseToolkit
from langchain.agents.agent_types import AgentType
from langchain.agents import create_sql_agent
import cx_Oracle
import os
import cohere
import os
COHERE_API_KEY="Your Cohere API Key"
os.environ["COHERE_API_KEY"] = COHERE_API_KEY
from sqlalchemy import create_engine
engine=create_engine(url, echo=True)
db = SQLDatabase(engine)
lib_dir = os.path.join(os.environ.get("HOME"), "Development", "instantclient_19_8")
cx_Oracle.init_oracle_client(lib_dir=lib_dir)
hostname='localhost'
port='...'
service_name='...'
username='<...>'
password='<...>'
# cx_Oracle.init_oracle_client(lib_dir=lib_dir)
oracle_connection_string_fmt = (
'oracle+cx_oracle://{username}:{password}@' +
cx_Oracle.makedsn('{hostname}', '{port}', service_name='{service_name}')
)
url = oracle_connection_string_fmt.format(
username=username, password=password,
hostname=hostname, port=port,
service_name=service_name,
)
agent_executor = create_sql_agent(
llm=Cohere(temperature=0),
toolkit=SQLDatabaseToolkit(db=db, llm=Cohere(temperature=0)),
verbose=True,
agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
)The way that Langchain works is that you can configure it as a library inside of your application and connect it to different endpoints.
In our example, we connected to a database and an API endpoint. Still, we've also got a large language model, which gives us our natural language capability to query the database using natural language.
We might ask the LLM, "How many Materials did we create yesterday?"
The application receives the prompt "How many Materials did we create yesterday?" and sends that through LangChain.
LLMs can't execute SQL Queries; LLMs are text encoders or decoders, not SQL executors. Here is where we interact with LangChain Agents. Agents will receive the SQL query generated and will execute the SQL query against the database to get the information.
The whole thing is about prompt engineering; we did in-context learning. I just introduced how it would work with an SQL query, but this can be applied to many other topics; we can use Agents to execute Matemathic calculations (LLMs don't do mathematics; they are text encoders and decoders), they can book a flight for us if they can connect to any SAP API from any other application, which I will describe in other blogs.
LlamaIndex alternative
While Langchain is a framework for Generative AI application development and orchestration, LlamaIndex (formerly GPT Index) is a data framework for LLM applications to ingest, structure, and access private or domain-specific data.
With LlamaIndex, thanks to its data connectors, you can effortlessly incorporate data from diverse sources such as APIs, databases, and PDFs. This data is structured into optimized intermediate formats suitable for LLMs. LlamaIndex enables seamless interaction and conversation with your data through query engines, chat interfaces, and LLM-powered data agents, all in natural language.
LlamaIndex vectorized the data, as we described in previous blogs, to prepare the data into a format that can be understood by the LLM. The process primarily encompasses two key phases: the indexing phase and the querying phase. The primary purpose of using LlamaIndex is the standardization of different sources and the performance increase this will provide.
Conclusion
In this blog, we introduced the concept and some details of using LangChain’s SQL Database Chain and Agents with large language models to perform natural language queries (NLQ) of any Phyton SQLAlchemy database. I wanted to emphasize Agents, a fundamental piece in all modern frameworks; LangChain uses Agents, LlamaIndex uses Agents, and Bedrock just introduced Agents.
Using LangChain’s SQL Database Chain and SQL Database Agent, we can leverage large language models (LLMs) to ask questions of multiple types of databases using natural language without building the query ourselves. Questions will be converted into SQL queries and executed against the database. Assuming the generated SQL query is well-formed, the query results will be converted into a textual explanation. For example, we ask questions like, “How many customers have purchased this Material in the last 12 months?” or “What were the total purchases we had in August for this company code?” These will be converted into SQL SELECT statements. The answer is then composed into textual explanations as a response to our application.
- SAP Managed Tags:
