Automated Business Marketing Consultant AI Agent

Exploring Multi-Agent Generative AI: An exploration into Language Model Composability, Research & Development, Tokenization, Vectors, Synthetic Data, and Google Cloud Deployment

Approximately six months ago, Chat GPT made a dramatic entrance to the AI scene. What was and is happening in the ecosystem around it has been bustling with activity equally if not more exciting. The pace is relentless and keeping up with all the developments is a formidable task.

For me, the potential of AI has always captivated me. Despite having a truly modest background in development, I’ve been exposed to and fascinated by technological advancements from an early age. With a vast array of information at my disposal thanks to the internet and Transformers, I feel like an excited child in a candy store — or more accurately like I’ve been given a swiss army knife with the frontiers of knowledge on it’s sharp edges.

A friend and I were discussing our respective interests in the various intersections of machine intelligence and other fields. During this conversation we imagined a project designed to broaden my AI product management skills and enhance his technical and development expertise in AI.

We are both of the opinion that solutions associated with the challenges of widescale deployment of these technologies will come from integrating safety as an assumption into product development. So taking this lens into the project made sense for us. You can see this is the case at leading firms like Anthropic where safety is of paramount concern.

Forward three weeks, my friend and I decided to work on an end-to-end project that spanned from ideation to development.

While the project was not particularly aimed at solving a specific problem, it was gratifying to see our concept function as intended.

To create “duelling agents” that discussed and developed a marketing plan for an arbitrary startup solely based on the name and industry. The non-determinism of LLM’s made the outputs particularly interesting.

Insights Gained from the Project

This project allowed me to familiarize myself with several new concepts:

Composability with Large Language Models (LLM) through “Langchain”, the “Ansible of LLMs.” Still only scratching the surface with this one.
Development of complex information flows and the utilization of research papers to understand the theoretical foundations of our objective.
Tokenization.
Vector embeddings.
Vector storage and querying.
Creation of a front-end User Interface (UI) using Streamlit.
Cloud deployment and containerization.

Notes:

We didn’t give the agents external tools which is easy to do for web searching etc. This is mostly because of the extensive parsing and data processing required to get the information into something passable. It was judged the gain was not worth the effort for a proof of concept.
We didn’t parse the outputs from the LLM’s or separate out the different questions formally. The LLM made the decisions about which direction to take the conversation and it varied across examples.
I’m curious about Weights & Biases as a tool and hope to use it in a simpler project to become familiar with tracking and metrics “LLMOps.”

The Journey

Our project underwent five iterations, each of which provided valuable lessons. Each time the flaws in approach would become quickly apparent. In addition the flaws in my debugging were flown directly in my face when an hour on a problem was found to be abusrdly simple but I was flogging the wrong horse. As far as LLM specific issues go, control flow could be a challenge. Occasionally, the AI agents would get stuck in a farewell loop, but we managed to bring it to an acceptable level of functionality.

Where the CAMEL?

The inspiration and final solution was to implement the conversational agents of Camel-ai.org, which work cooperatively to achieve a goal.

The authors of the original paper were investigating the emergence of new properties during ongoing conversation with memory. I also wanted to explore vector embeddings and Pinecone vector storage for similarity scores and to evaluate these technologies as part of a simple proof-of-concept.

Moreover, I used Gretel’s generative AI to produce additional data and a model capable of generating records similar to the unstructured conversation, in theory. The presence of some differentiation within the files boosted my confidence that the model Gretel used for training on my data could detect some pattern.

Ideally, we would have a model capable of responding to queries about a specific startup name or industry vertical by providing “similar responses” or “common responses in high-dimensional space.”

Initial Attempts and Missteps

Here are some of the mistakes and lessons from my early attempts:

Using Langchain in a linear format to link together agents and their outputs resulted in dull progressions due to rudimentary memory application and lack of a feedback loop. Single shot agents in particular were weak as you’d expect.
The discovery of AutoGPT and similar tools highlighted the importance of enabling individual units to perform operations or evaluate information independently and repetitively to produce a legible outcome.
My attempts to build these out with my limited programming skills had some success, but the outcome was not inspiring.
I then discovered the CAMEL implementation, which I highly recommend exploring.

How the Final Version Works End-to-End

The user inputs a startup name and an industry vertical description. Beyond this, the user doesn’t interact with the system.
Two classes of the CAMEL helper from Langchain are initialized and prompted with the task. This required careful crafting and adjustment of the prompts.
The “Startup Founder” asks the assistant who will continue editing the user’s text.
the “Marketing Consultant” for help. Each response from either party is fed into the GPT 3.5 Turbo model, chosen for its cost efficiency and accessibility.
The conversation continues until the Large Language Model (LLM) determines that the goal has been achieved, at which point the assistant agent—our marketing consultant—issues the final statement: “.” This feature is incredibly intriguing.
During each iteration, the output is saved as raw text to a text file for further processing.
After the conclusion of the “conversation,” the file is passed through the embeddings endpoint for tokenization. The process of tokenization involves converting the words or sub-words into numeric representations, depending on the tokenizer used.
Vector embeddings: The embeddings endpoint from OpenAI, using the Ada model, transforms these chunks of information into 1536-dimension vectors for machine interpretation. Essentially, this is the mathematical representation of a subword in high-dimensional space. An example of a vector is below in Appendix A.
The set of vectors is then “upserted” to Pinecone using the Pinecone client and stored in its entirety. While vector storage has been employed by large companies for massive datasets for years, this modest use case aims to enhance my understanding of the concept. Traditional search methods may prove more effective once the data is structured better, but I have yet to evaluate this.
After uploading the 20 or so files created using various business names, queries can be made on the data.

The Role of Synthetic Data
Around this time, I encountered the concept of “synthetic data,” a method for expanding your datasets for training within a machine learning pipeline. Your data, regardless of its format, is augmented by the system. An example used was unstructured Taylor Swift lyrics, which inspired me to experiment with my text files.

I wondered if I could extend my 20 generated conversations to a larger number relevant to business marketing. Because of the unstructured text nature of the documents it required some pre-processing and I am wondering if some of langchains chunking or other processing tools could be of benefit.

Finally, I am proceeding under the assumption that the quality is perhaps proof of concept level but I wanted to explore containerizing the application for uploading to Google Cloud Platform. Right after building a simple Streamlit front end with a chat like interaction. This UX element presents a challenge. Langchain does allow intermediate steps to be output but I am not across this as yet.

Once deployed, I am eager to observe how the different components work together. The application itself from within it’s container, how and if it can interfact with Pinecone as expected.

Future Improvements

The use of web search and tools like BeautifulSoup to process up to date market information.
Improved prompting through comprehension of how the two interplay.
Better planning initially. Vision planning in particular. I guess, in reality this point speaks to testing the project more like an actual user problem – not a simple script being developed to solve nothing. In the spirit of the goal!
Pre-Query the Vector store for similarity between companies to save on LLM API fees.

It’s hard to overstate how thrilling this entire process is when all the actions come together. And I’m just one very average dabbler. Teams equipped with robust Data Science and Development capabilities have immense potential to harness these tool’s power. The possibilities are both thrilling and somewhat daunting.

In conclusion, I gained considerable knowledge and spent a great deal of time pondering the implications. Despite dabbling in programming numerous times since the age of 11, I still feel like a beginner. While I understand the individual components, I lack the recipes for complex projects, which unfortunately are my favorites. Nevertheless, the journey is incredibly enjoyable.

Here are some relevant articles that delve deeper into the concepts covered:

Spotify’s Podcast Search Explained | Pinecone
camel-ai.org
pinecone.io
python.langchain.com (though there is a JavaScript SDK available)
gretel.ai (synthetic data)
MostlyAI (synthetic data)
Github Repo

Below is a summary of my code, as interpreted by GPT-4 via simply putting the code block into GPT-4:

The provided code is a Python script that leverages the OpenAI GPT model to create an interactive business agent that helps to develop marketing strategies for businesses. The primary module in this script is the CAMELAgent class, which manages the conversation between the user and the AI agent. The agent uses a step function to process and update messages, and its purpose is to follow an iterative pattern of prompting and responding to the user’s queries, advancing the conversation in a sequential manner. Additionally, the agent can be reset, enabling it to begin the process afresh.
The bulk of the script defines the bus_agent function, which creates an interactive session between a startup founder (the user) and a market research consultant (the AI). The agents begin by discussing the business idea and passing unique ideas back and forth. The founder can instruct the consultant, who can only provide advice in response. The conversation progresses with the agents alternatively asking and responding to questions about marketing strategy and target market. These dialogues are saved to a file, and the session ends when the founder sends a “” message. The function also includes generating a summary of the chat and business. It additionally enables the agents to write a 2 paragraph summary outlining the product and market it fits into.
The final portion of the script sets up a Streamlit UI for a more interactive and friendly user interface. The UI allows the user to input their OpenAI API key, the name of their business, and a description of their industry. This data is then passed to the bus_agent function to initiate a conversation. The results of the conversation, as well as any past conversations, are stored in the session state and displayed on the Streamlit page. The script provides the functionality of an AI-based business consultant that assists in the development of marketing strategies for startups.
– GPT4

Embedding & Vector Storage
Summary
The Python code provided is used for splitting, embedding, and storing text data in a Pinecone vector store for later querying. The program imports various libraries and modules for handling environment variables, managing progress bars, creating unique identifiers, tokenization, loading texts, splitting texts, creating embeddings, and interfacing with Pinecone. The implementation is divided into three primary parts:
1 Configuration: This portion loads necessary environment variables (e.g., API keys) and initializes global components like the Pinecone client, a tokenizer, and an embedding model.
2 Embedding and Storing: The function embed_upsert is defined to read, split, embed, and store text data. It takes in a filename, business name, and industry as arguments, then reads the file and splits the text into chunks. Each chunk of text is then embedded and upserted (i.e., updated or inserted) into a Pinecone index along with associated metadata. This function is designed to handle large file sizes by batching upsert operations.
3 Querying: The query_vec function is used to perform similarity search queries on the Pinecone index. It loads and splits text documents, creates a Pinecone index from these documents, and performs a similarity search for a given query.
4 Embedding a Folder of Files: The function embed_folder is defined to handle multiple files in a folder. It lists all files in the given folder and processes them one by one using the embed_upsert function.
One point to note is that the outputs are stored as .txt files initially. There is a mention of adding a query to the summary, but it is not clear how the query should be incorporated in the context of the provided script.
-GPT4

Appendix A:

objects embedded into vectors and then similarity shown in 2D vs 7D.