StructuredOutputParser seems verbose - heres another implementation

[bradennapier]

I have had perfect results thus far with a far less verbose method which doesn't require all the extra cognitive additions and formatting / token costs.

It works by using the typescript compiler to take a zod schema and turn it into a string with jsdoc if describe is used. It has even worked flawlessly with unions, intersections, etc (for example something like { success: true } | { success: false; reason: string }. Just thought I'd share

Zod Schema / Result

z.object({
      langs: z.tuple([z.string(), z.string()]).describe('[From, To] as proper names'),
      from: z.literal('I love programming.'),
      to: z.string().describe('The message translated'),
      n: z.number().describe('The number of words in the translated text'),
})

It works without the from: being a literal, I just added it for extra safety.

{
  "langs": [
    "English",
    "Spanish"
  ],
  "from": "I love programming.",
  "to": "Amo programar.",
  "n": 2
}

Perhaps the concern is the AI will sometimes misbehave, however I have found the language used to be very reliable with my testing.

Generated Prompt Example

You are a helpful assistant that translates English into Spanish.

type OutputJSON = {
    /** [From, To] as proper names */
    langs: [
        string,
        string
    ];
    from: "I love programming.";
    /** The message translated */
    to: string;
    /** The number of words in the translated text */
   n: number;
};

You are a text transformer that will take the input from the user and reply with type OutputJSON as JSON.
Only reply with JSON, never english

ChatPromptValue {
  messages: [
    SystemChatMessage {
      text: 'You are a helpful assistant that translates English into Spanish.\n' +
        '\n' +
        'type OutputJSON = {\n' +
        '    /** [From, To] as proper names */\n' +
        '    langs: [\n' +
        '        string,\n' +
        '        string\n' +
        '    ];\n' +
        '    from: "I love programming.";\n' +
        '    /** The message translated */\n' +
        '    to: string;\n' +
        '    /** The number of words in the translated text */\n' +
        '    n: number;\n' +
        '};\n' +
        '\n' +
        'You are a text transformer that will take the input from the user and reply with type OutputJSON as JSON.\n' +
        'Only reply with JSON, never english',
      name: undefined
    },
    HumanChatMessage { text: 'I love programming.', name: undefined }
  ]
}

OutputParser

import type { z } from 'zod';

import { createTypeAlias, printNode, zodToTs } from 'zod-to-ts'
import { BaseOutputParser, OutputParserException } from 'langchain/schema/output_parser';

const identifier = 'OutputJSON'

export class JSONOutputParser<
  T extends z.Schema
> extends BaseOutputParser<z.TypeOf<T>> {
  /**
   * The Zod schema parsed into a typescript type 
   * which is used to feed the expected output
   * to the AI.
   */
  public formatted: string;

  constructor(public schema: T) {
    super();
    const { node } = zodToTs(this.schema, identifier)
    const alias = createTypeAlias(
      node,
      identifier,
      // optionally pass a comment
      // comment: UserSchema.description
    )
    this.formatted = printNode(alias)
  }

  static fromZodSchema<T extends z.Schema>(schema: T) {
    return new this(schema);
  }

  getFormatInstructions() {
    return `
${this.formatted}

You are a text transformer that will take the input from the user and reply with type ${identifier} as JSON.
Only reply with JSON, never english`;
  }

  async parse(text: string): Promise<z.TypeOf<T>> {
    try {
      const result =  this.schema.parseAsync(JSON.parse(text.trim()))
      return result as Promise<z.TypeOf<T>>
    } catch (e) {
      throw new OutputParserException(
        `Failed to parse. Text: "${text}". Error: ${e}`,
        text
      )
    }
  }
}

Implementation

export async function chainChat() {
  const chat = new ChatOpenAI({ temperature: 0 });
  
  const parser = JSONOutputParser.fromZodSchema(
    z.object({
      langs: z.tuple([z.string(), z.string()]).describe('[From, To] as proper names'),
      from: z.literal('I love programming.'),
      to: z.string().describe('The message translated'),
      n: z.number().describe('The number of words in the translated text'),
    })
  )

  const prompt = await prompts.chat.translate({
    from: 'English',
    to: 'Spanish',
    message: 'I love programming.',
    formatInstructions: parser.getFormatInstructions()
  });

  console.log(prompt);

  const output = await chat.callPrompt(prompt);

  console.log(output)

  const result = await parser.parse(output.text);
  console.log('Result:\n', result)
}

Result:
{
  langs: [ 'English', 'Spanish' ],
  from: 'I love programming.',
  to: 'Amo programar.',
  n: 2
}

[bradennapier]

Could pretty easily allow defining custom formatting that enforces the write name for the type so that it is easier to create advanced type formats (but also easier to diverge from the zod schema) by just accepting an optional format string strictly types to at least have type OutputJSON = { ... }

const identifier = 'OutputJSON' as const
export type CustomFormatString = `${any}type ${typeof identifier} = {${any}}${any}`;

It has worked for far larger types like

Template

type Output ={
  input: Input;
  character: {
    firstName: string;
    lastName: string;
    age: number;
    gender: string;
    ethnicity: string;
    personalityTraits: {
      openness: number;
      conscientiousness: number;
      extraversion: number;
      agreeableness: number;
      neuroticism: number;
    };
    personalityDescriptions: string[]; // Array of descriptions about their personality
    hobbies: string[];
    values: string[];
    lifeGoals: string[];
    fears: string[];
    strengths: string[];
    weaknesses: string[];
    backstory: string;
    lifeStoryDescriptions: string[]; // Array of descriptions about their life story
    relationships: {
      family: string[];
      friends: string[];
      enemies: string[];
    };
    currentLifeSituation: string;
  };
};

Output

{
  "input": "The most powerful wizard that ever lived who has a magical aura",
  "character": {
    "firstName": "Merlin",
    "lastName": "",
    "age": 943,
    "gender": "Male",
    "ethnicity": "",
    "personalityTraits": {
      "openness": 0.8,
      "conscientiousness": 1,
      "extraversion": 0.4,
      "agreeableness": 0.5,
      "neuroticism": 0.3
    },
    "personalityDescriptions": [
      "Merlin is incredibly wise and knowledgeable, with a deep understanding of magic and the mystical realm. He is also highly disciplined and hardworking, constantly pushing himself to achieve new heights in his powers.",
      "Despite his great power, Merlin is generally reserved and introspective, preferring to spend much of his time alone in meditation or deep contemplation of the mysteries of the universe."
    ],
    "hobbies": [
      "Studying ancient texts to unlock new spells and understand the secrets of the universe.",
      "Meditating on the nature of existence and consciousness.",
      "Occasionally traveling through time to study different eras and learn from their cultures."
    ],
    "values": [
      "The pursuit of wisdom and knowledge above all else.",
      "Maintaining balance in the universe through careful use of his powers.",
      "Teaching others about magic and helping those who seek his guidance."
    ],
    "lifeGoals": [
      "To continue learning and growing in his magical abilities, eventually unlocking the full potential of his powers.",
      "To maintain balance in the universe and keep dark forces at bay.",
      "To share his knowledge with others through teaching and mentoring young wizards."
    ],
    "fears": [
      "Losing control of his powers and causing harm to the world through unintended consequences.",
      "Becoming corrupted by power and turning to the dark side of magic.",
      "Being isolated from others for too long and losing touch with reality."
    ],
    "strengths": [
      "Unparalleled mastery of magic, with a deep understanding of even the most obscure spells and rituals.",
      "A profound sense of wisdom and insight, allowing him to see beyond the superficial surface of reality.",
      "A deep connection to the natural world and the mystical realm, granting him great power over the forces of nature."
    ],
    "weaknesses": [
      "Tendency to become so absorbed in his studies and meditation that he neglects his physical health or social life.",
      "Occasional bouts of intense self-doubt or anxiety, particularly when faced with daunting challenges or enemies.",
      "Limited ability to relate to others who do not share his deep understanding of magic and mysticism."
    ],
    "backstory": "Merlin was born into a family of powerful wizards, each specializing in a different aspect of magic. From a young age, it was clear that Merlin was destined for greatness, manifesting astounding magical abilities even as a small child. He studied under some of the greatest wizards in history, eventually surpassing them all with his exceptional powers. Over time, Merlin began to take on the role of teacher himself, mentoring young wizards and sharing his knowledge with those who sought it out. He has witnessed many great events over the centuries, from wars between kingdoms to earth-shattering cosmic events. Throughout it all, he has remained dedicated to his pursuit of knowledge and wisdom, never faltering in his quest to unlock new secrets about the universe.",
    "lifeStoryDescriptions": [
        "Throughout his long life, Merlin has been instrumental in many pivotal moments throughout history, from quietly advising kings and emperors to directly intervening in battles between powerful wizards.",
        "Merlin has also spent much of his life pursuing personal growth and enlightenment, venturing deep into the mystical realm to explore the furthest reaches of the universe.",
        "While Merlin is often seen as remote and aloof by those who do not know him well, those who encounter him on a deeper level come to understand the profound compassion and empathy that underlie his actions."
      ],
    "relationships": {
      "family": [],
      "friends": [
        "King Arthur, whom he mentored throughout his youth and later advised during many important battles.",
        "Morgan le Fay, a powerful sorceress with whom he has engaged in many epic magical battles over the centuries.",
      ],
      "enemies": [
        "Mordred, a dark wizard who seeks to overthrow the kingdom and plunge it into chaos.",
        "Vortigern, a powerful warlock who has long sought to capture and steal Merlin's magic for his own uses."
      ]
    },
    "currentLifeSituation": "Merlin spends most of his days secluded in his tower deep within an enchanted forest, meditating on the nature of reality and continuing to unlock new secrets about the mystical realm. However, he occasionally emerges from his hermitage in order to advise important leaders or intervene in crises affecting the balance of power in the universe."
  }
}

[deepakkashyap3013]

hey @bradennapier, I need some help, whenever I am executing the code with some input string to query my database I am getting this error, it would be helpful if you guide me if I am doing something wrong here, the below is the error and the code that I am executing...I suspect after this particular line the error appears : const formatInstructions = parser.getFormatInstructions(); can be seen from the code snipet

Find me the challenge and weight from votes table
formating........................
/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parsers/array.js:13
        res.items = (0, parseDef_1.parseDef)(def.type._def, Object.assign(Object.assign({}, refs), { currentPath: [...refs.currentPath, "items"] }));
                                                      ^

TypeError: Cannot read properties of undefined (reading '_def')
    at parseArrayDef (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parsers/array.js:13:55)
    at selectParser (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parseDef.js:101:46)
    at parseDef (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parseDef.js:42:24)
    at Object.assign.Object.assign.Object.entries.reduce.properties (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parsers/object.js:10:51)
    at Array.reduce (<anonymous>)
    at parseObjectDef (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parsers/object.js:7:96)
    at selectParser (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parseDef.js:89:48)
    at parseDef (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/parseDef.js:42:24)
    at zodToJsonSchema (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/zod-to-json-schema/src/zodToJsonSchema.js:16:48)
    at StructuredOutputParser.getFormatInstructions (/home/cynefin/Documents/FULL STACK APP/Exercise Files/react-ecosystems-server/node_modules/langchain/dist/output_parsers/structured.cjs:37:59)

Node.js v18.15.0

here is my code

import { DataSource } from "typeorm";
import { z } from "zod";
import { OpenAI } from "langchain/llms/openai";
import { SqlDatabase } from "langchain/sql_db";
import { createSqlAgent, SqlToolkit } from "langchain/agents";
import { PromptTemplate } from "langchain/prompts";
import { StructuredOutputParser } from "langchain/output_parsers";

let AIresult;

export const AIassistant = async (inputString) => {

    const datasource = new DataSource({
        type: "postgres",
        host: "localhost",
        port: "5432",
        username: "username",
        password: "password",
        database: "mydatabase",
        schema: "schema_name",
    });

    const db = await SqlDatabase.fromDataSourceParams({
        appDataSource: datasource,
    });

    const parser = StructuredOutputParser.fromZodSchema(
        z.object({
            outputArray: z.array().describe("should be an array of objects"),
            SQLquery: z.string().describe("should be a SQL query in form of string")
        })
    );

    console.log("formating........................")
    const formatInstructions = parser.getFormatInstructions();
    console.log(formatInstructions);
    const prompt = new PromptTemplate({
        template:
            "You are an expert at SQL querying, answer the users question as best as possible.\n{format_instructions}\n{question}",
        inputVariables: ["question"],
        partialVariables: { format_instructions: formatInstructions },
    });

    const model = new OpenAI({
        temperature: 0,
        modelName: "gpt-3.5-turbo",
        openAIApiKey: "sk-xxxxx...",
    });
    const toolkit = new SqlToolkit(db, model);

    const executor = createSqlAgent(model, toolkit);

    const input = await prompt.format({
        question: inputString,
    });

    const result = await executor.call({ input });

    await datasource.destroy();

    AIresult = await parser.parse(result.output);

};

export const returnResult = () => {
    return AIresult;
}

[bradennapier]

That is a question. For zod to schema package which I've never used and removed from my solution in the OP but it's because you just did z.array() if I were to guess the json scheme wants to assign something to the items of the array.

I'm not 100% but I'd look at zod docs and try something more like z.array(z.record(z.string(), z.any())) if z.any() exists or equivalent.

That being said my option above would prob work as you defined it lol although it'd accept any array not just an array of objects if so.

[deepakkashyap3013]

Thanks, for your response, will try to do the necessary changes, see if it works out