Scaffolding Around Models: The Primary Barrier to Production Use

It is evident to regular users of artificial intelligence (AI) that current models exhibit a level of intelligence comparable to that of a smart high school student or college freshman.

These models possess the potential to automate numerous jobs. However, their limited adoption in production environments is not due to a deficiency in raw intelligence. Rather, it is the lack of comprehensive user experience (UX) and product features, including integrating relevant contexts for company-specific information (such as chats, documents, and codebases), improving voice features for seamless conversations, and enabling models to participate and reliably interact with the company’s existing tools and software, that hinders their full utilization.

These advancements are anticipated to occur alongside a significant increase in computational power applied to AI models over the next 18 months, paralleling the progress observed from GPT-2 to GPT-4. The following developments are expected in this period:

1. Enhanced retrieval-augmented generation (RAG) practices for integrating relevant contexts (retrieval, embeddings, reranking). This involves a hybrid search layer to identify potential context chunks, extract and organize them, pull the relevant data from connected systems, and then combine it in an optimized request to the AI.

2. Accelerated voice model development (ElevenLabs)

3. Integration of multiple AI subsystems with clear objectives

4. Best practice frameworks for UX/UI and Co-Pilot integration derived from current production experiments: Users interact with the Co-Pilot by observing suggested goals, selecting and reviewing the workflow execution plan, approving it, and following along step-by-step, with options to adjust or interrupt the AI workflow as needed.

For more details on Multi-Agent LLM Frameworks check our post here.