Peviously written posts on AutoGen and MemGPT didn’t emphasize enough the profound implications of these projects on the AI scene. Let’s delve further into their significance and impact on the AI sphere.

Some insights:

  • AutoGen:
    • Recap: AutoGen is an open source project from Microsoft Research, is playing a pivotal role in enabling collaborative AI, free from the constraints of a single agent.
    • There are lots of AutoGen similar projects like (not exhaustive list) Chat Dev, Open Interpreter, Chat Dev, GPT Pilot/Engineer, Github Copilot (Enterprise), MetaGPT, AutoGPT, BabyAGI, HF Transformer Agents which all have their pros and cons, but there seems to be a common agreement within the AI scene that AutoGen rules them all.
    • AutoGen’s core mission is to facilitate the creation of multi-agent teams with a remarkable skill set that includes code writing and execution, offering feedback, and embodying various roles and personas.
    • While ChatGPT is a powerful AI, AutoGen takes AI collaboration to the next level. It addresses a critical issue: the efficiency of AI feedback loops. Working with a single AI often results in errors and inaccuracies, requiring human intervention for correction. AutoGen introduces a multi-agent framework, allowing another AI to provide feedback, shortening the feedback loop and reducing the need for human intervention. This automated feedback process is particularly vital in the realm of code writing.
    • A notable feature of AutoGen is its “drop-in replacement” capability for the ChatGPT API. This feature offers an array of valuable functionalities, including built-in retry logic, caching, model fallback, multi-model support, goal setting, budgeting, and more. Even if you don’t plan to use AutoGen for building AI agent teams, this API wrapper alone is worth considering.
    • AutoGen’s support for open-source models adds another layer of flexibility. While the project’s complexity often demands the power of GPT-4, AutoGen allows for the assignment of specific models to agents. This paves the way for diverse AI capabilities, ensuring that AutoGen remains adaptable as open-source models continue to evolve.
    • AutoGens capabilities reach far beyond coding tasks, an example of Use Cases – AutogenAI you can dedicate your AIs (AutoGen roles defined) to:
      • Bids & Proposals: define optimized strategy to win a contract.
      • Marketing: Create compelling stories.
      • Sales Materials: Craft tailored sales materials.
      • Thought Leadership: Develop blogs, white papers etc.
      • Internal Comms: Ensure consistent employee communication.
      • Public Relations: Write impactful press releases.
  • MemGPT:
    • Researchers from UC Berkeley developed this  novel technique, which teaches LLMs to manage their own memory using virtual context management.
    • Addresses the memory limitations of LLMs, thereby (possibly) offering unbounded context.
    • Knows when to push critical information to a vector database and when to retrieve it later in the chat, enabling perpetual conversations.
    • Makes an LLM chatbot not just a tool, but a companion/copilot that constantly learns and involves.
    • Features like memory retention and user feedback integration contribute to its growth.
    • Signifies a future where AI focuses on relationships and conversations, not just responses.
    • ! MemGPT just implemented AutoGen, of which an example below.
      The green, red, and orange are the AutoGen LLMs, and the small clouds are where MemGPT comes in and “thinks”:
  • A combination of AutoGen and MemGPT: as the screenshot above exemplifies: you now not only have a battalion of LLMs figuring out things for you. They now “know” your personality/preferences/history and thereby minimize (or even eliminate -futurewise-) feedback loops and/or any user input.
  • Bringing in Open Source and an “Uber GPT-X”: AutoGen, with its GPT-4 integration, can become very costly pretty quickly. Specifically, since internal loops might occur. MemGPT might cover parts of that, but running local/open-source LLMs is free, and that’s only one of the plenty of other reasons to run (local) LLMs. Though the downside of open-source is their performance versus GPT-X (4 for now). So, as per my feature request to AutoGen, combining AutoGen Open Source LLMs with GPT-4 might be “a great idea” which AutoGen developers appear to support, and I will certainly explore this pathway (using a function call) further on <screenshot below>.

    ! Note that, for AutoGen to be performant with open-source LLMS, you would need to have the most performant (local) open-source LLM(s) possible, as that will be the weakest point in the LLM chain. Compare the setup with having a 4-year-old (lower IQ, weaker open-source LLM) versus a more performant one (higher IQ, 18-year-old) which asks the savant (GPT-4, via function call) the right questions.