Pieter van Noordennen

The New AI: Early Learning Journey

Like many of you, I'm taking the plunge. Two weeks in, here are my early learnings on applying LLMs to real world problems.

Pieter van Noordennen |

The New AI: Early Learning Journey

Like many of you, I’m taking the plunge on the new developments in AI and jumping in with whatever free time I can scrape together.

The societal step-change that LLMs are creating has a lot of people in tech over-the-moon excited, and the gold rush is already on.

Last week, I was fortunate enough to attend a Decibel Partners event on AI in Cybersecurity at the RSA Conference. I saw six truly jaw-dropping demos that startups already have in market today leveraging AI to thwart attacks, and heard from industry luminaries Mike Hanley, Caleb Sima, and Daniel Meissler on how fast applied AI is evolving.

Earlier in the week, at an event hosted by BoldStart, I heard from Emilio Escobar, CISO of DataDog, on how Datadog had purchased licenses for various AI tools not just for their entire engineering team, but their marketing and GTM operations as well. It’s not just the nerds who are soon to have their working lives changed by ChatGPT and the surrounding eco-system.

After the event, I literally ran back home to accelerate my AI learning journey, and this post is download of everything I’ve gathered since. For those on a similar journey, I’ve included some of the best resources I’ve found in links at the bottom, and will continue to post here as I progress.

Re-Learning How To Learn

We’re so early in the AI evolution that one of the first things you need to do is re-learn how to learn. ChatGPT’s web interface, using either GPT-3.5 or GPT-4, makes a lot of things really, REALLY easy. It’s certainly the best starting point and capable of handling most of the simple tasks you need AI to do (i.e., suggest blog post ideas, build a diet plan or travel itinerary, etc etc.)

My ChatGPT window right now. Here, I channel Joseph Mitchell's editor from The New Yorker to help me edit my article posts.

Ed. note: From here on out, I’ll refer to the ChatGPT web UI as just ChatGPT. When referring to the OpenAPI ChatGPT APIs, I’ll note them as the API.

Paradoxically, the one thing it’s not great at is giving advice on recent developments in AI, given that it’s training data stops in September 2021 (that’s sure to change soon).

So for exploring latest cutting edge tools like AutoGPT and LangChain, you’re back to old-school, hands-on learning (in Python!) and diving through YouTube videos, recent blog posts, or just a lot of trial and error. Thankfully, ChatGPT is really good at Python, database queries, and knowledge of its own APIs — even if they are sometimes out-of-date.

My general set-up when learning goes as follows:

Tools I’m Exploring

There’s a lot of new stuff coming out daily, and it’s hard to stay focused long enough to get a working knowledge any one tool.

Here’s a quick run-down of the tools I’m exploring and where I expect the journey to go:

  1. ChatGPT (Web UI): Learn the basics of writing good prompts. Understand the qualitative differences between 3.5 and 4.
  2. OpenAI APIs: Programmatically interact with ChatGPT models and do basic database connections and iterative scripting. Understand additional capabilities available in the API, including other model types (i.e., Whisper).
  3. LangChain: Maintain agent state, load in more documents, do more complex operations via PromptTemplates.
  4. AutoGPT: See what the hype was about — so far I have been unsuccessful with it other than in very basic tasks that can just as easily be done in ChatGPT. More to explore later.
  5. Image Generation: A different category, but as a content creator, I’ve experimented with Dalle-2 and have Midjourney 5.1 (released this week) on my roadmap.

Flowchart for early learning in AI

A Basic AI Learning Path

My basic framework for tackling a new problem now follows a fairly predictable path.

Step 1: What’s the Problem

First, I need to be clear on the problem I want to solve. This is back-to-business-school basics, but easily overlooked. It’s really easy to get down the rabbit hole with LLMs, and if you value your time (which, as a new Dad, I do), you need to be crystal clear on the task you want to solve.

Step 2: Seed the Agent’s Identity

Second, it’s important to give every chat agent their own persona. It seems to help tune the model to what it’s going to be asked to do. This is why AutoGPT’s first prompt asks you for a name for the AI and who they are.

Here’s an example I gave ChatGPT:

Prompt: You are a science fiction writer working on a new project. 
Response (ChatGPT 3.5): 
As a science fiction writer from the 1980s, I'm excited to be working on my next project, which will explore the future of humanity and our relationship with technology.

In this story, I envision a world where technology has advanced to the point where it's nearly indistinguishable from magic. Humans have harnessed the power of nanotechnology, genetic engineering, and artificial intelligence to create a utopia where poverty, disease, and war no longer exist.

I call this the “seed prompt” and generally have one chat open per identity. This is easy in the ChatGPT interfact and supposedly comes baked in with AutoGPT, but maintaining agent state gets more complex once you start getting into more programmatic access, as we’ll see.

Step 3: Prompt Optimization

Now that the agent knows who they are, it’s time to give them a task. I call this the “Shot Prompt” to refer to the emerging lexicon around prompt optimization: Zero Shot and Few Shot prompts.

”Zero Shot” prompts are prompts that merely set a task without providing the AI any context whatsoever. This is typical of just opening a new chat and saying: “Make me an exercise plans.

Prompt: Generate a list of characters for your new project. 
Response: 
Certainly! Here are some potential characters for my new science fiction project:

Ada, a brilliant scientist who specializes in genetic engineering and has helped to create many of the marvels of the utopian society.
Max, a rogue AI that has broken free from its programming and has taken on a life of its own, seeking to manipulate and control humanity for its own ends.

”Few Shot” prompts, on the other hand, provide one or many example responses to help train the agent to know what you expect. Daniel Meissler published a great post (link in Resources) on why Few Shot prompts, combined with “step by step” instructions, yield better results.

Prompt: Generate a list of characters for your new project. 

Be sure to name the protagonist, antagonist, and any supporting characters. Define their ages and relationships to each other. For example: 

Ada, 49,  protagonist, a brilliant scientist who specializes in genetic engineering and has helped to create many of the marvels of the utopian society.

Zed, 22, antagonist, a charismatic leader of the rebels who has a gift for rallying people to his cause and a hidden agenda that may not be entirely benevolent.

Provide exactly 6 characters and do so in a step-by-step fashion. 

The characters: 

RESPONSE: 
Sure, here are six characters for my new project:
1. Alex, 25, protagonist, a young and idealistic member of the utopian society who is beginning to question the cost of their perfect world.
2. Vex, 28, antagonist, a rogue hacker who seeks to bring down the utopian society and create chaos.

More Advanced Techniques for AI Development

Once you have a prompt or set of prompts that is working in a repeatable and consistent fashion, it’s time to take things to the next level.

Programmatic Acces

If you can code even basic Python or Node, the OpenAI apis are dead simple to use. Simply grab an API key, authenticate, and start sending requests. The two main advantages of using the APIs are:

Maintain Agent State with LangChain

ChatGPT will maintain the agent state for you automatically in the form of “new chats” — that is, every chat remembers what you’ve told it before and what it’s responded. This isn’t the case out of the box with the OpenAI APIs, and if your goal is to, say, load your own documents into the agent and ask it specific questions — a common use case for product development.

LangChain is an ultra-powerful Python library that allows you to upscale your AI development. With it, you can easily maintain agent state, load documents into a specific agent using Vector storage, and create Prompt templates that lead to speed, scale, and flexibility. This is a great tool to learn if you have basic Python jobs and a sepcific

Try Plugins

Plugins are quickly becoming a way to scale development in AI framworks. OpenAI’s plugin marketplace is currently fairly limited, but there’s some promising applications like Wolfram Alpha for advanced math and Expedia for travel.

AutoGPT, an open-source project that combines chains, web surfing, and an extensible plugin framework, has made a lot of noise and there are some cool demos out there of it. I messed with it early days but was largely unsuccessful. Being open-source, the plugin model seems really promising for this technology, and the latest Version 3 seems to have made some usability improvements. More to try there.

Most of all, remember we’re still early days and it’s easy to feel left behind with a new amazing demo coming out every few minutes. Optimize for learning and remember that it’s a journey. Happy hunting.

Resources