Skip to Content
📖 Guide Documents📚 Core LibraryLLM Vendors

LLM Vendors

src/main.openai.ts
import {
  Agentica,
  IAgenticaController,
  IAgenticaProps,
  IAgenticaVendor
} from "@agentica/core";
import OpenAI from "openai";
import typia from "typia";
 
import { BbsArticleService } from "./services/BbsArticleService";
 
const agent: Agentica = new Agentica({
  vendor: {
    model: "gpt-4o-mini",
    api: new OpenAI({
      apiKey: "********",
    }),
  } satisfies IAgenticaVendor,
  controllers: [
    {
      protocol: "class",
      name: "bbs",
      application: typia.llm.application<BbsArticleService>(),
      execute: new BbsArticleService(),
    } satisfies IAgenticaController,
  ]
} satisfies IAgenticaProps);
await agent.conversate("I wanna buy MacBook Pro");

When creating Agentica instance, you can specify the LLM service vendor.

Agentica is utilizing OpenAI SDK (npm i openai) for LLM (Large Language Model) service interaction. However, it does not mean you can use only the OpenAPI’s GTP models. The OpenAI SDK is just a connection tool to the LLM vendor’s API, and as the most of modern LLMs are following the OpenAI’s API design, you can use other LLM vendors like Claude, DeepSeek, and so on.

In that case, configure IAgenticaVendor.api.baseURL and IAgenticaVendor.model properties to the LLM vendor’s API endpoint and model name. For example, if you want to use Meta Llama instead of OpenAI GPT, you can do it like below.

Schema Specification

You know what? JSON schema models are different between LLM vendors.

OpenAI GPT and Google Gemini have their own JSON schema models that are different from each other, even not following the standard JSON schema specification. @agentica and its underlying libraries (@samchon/openapi and typia) automatically handle these schema model differences internally, so you don’t need to specify the schema model explicitly anymore.

Besides, Anthropic Claude, HF DeepSeek, and Meta Llama are following the standard JSON schema specification (JSON schema 2020-12 draft, OpenAPI v3.1 is based on it). So, (IClaudeSchema, IDeepSeekSchema, ILlamaSchema) are actually the same as ILlmSchemaV3_1.

Standard JSON Schema

@samchon/openapi/IClaudeSchema
404: Not Found

In the major LLM vendors, (Anthropic Claude, HF DeepSeek and Meta Llama) are following the standard JSON schema specification (JSON schema 2020-12 draft, OpenAPI v3.1 is based on it). So, (IClaudeSchema, IDeepSeekSchema, ILlamaSchema) are actually the same type with ILlmSchemaV3_1.

In actually, only Anthropic Claude has announced that they are supporting the full specification of JSON schema 2020-12 draft. DeepSeek has not documented it yet, but there was not any problem when we tested it. Meta Llama does not support tool calling feature, so that it is based on prompting logic, and no explicit schema model specified, so no problem either.

In the same reason, if you take some other LLM vendor like Mistral AI , which has not specified any JSON schema model, you can use ILlmSchemaV3_1 as well, and it is actually working fine.

ChatGPT Schema

src/chatgpt.parameters.ts
import typia, { tags } from "typia";
 
typia.llm.parameters<{
  /**
   * Primary Key.
   * 
   * Primary Key generated by System.
   */
  id: string & tags.Format<"uuid">;
 
  /**
   * Age of the user.
   * 
   * Age of the user in years.
   */
  age: number & tags.Type<"uint32"> & tags.Maximum<100>;
 
  /**
   * Email address of the user.
   * 
   * Email address written when membership joining.
   */
  email: string & tags.Format<"email">;
}>();

OpenAI has its own JSON schema model different from the standard JSON schema specification.

OpenAI’s JSON schema model is based on JSON schema 2020-12 draft (OpenAPI v3.1 is following), but it has banned some constraint properties like IJsonSchema.IString.format and IJsonSchema.IInteger.minimum. In detailly, these specifications are not supported in the OpenAI’s JSON schema model IChatGptSchema.

  • IJsonSchema.INumber.minimum
  • IJsonSchema.INumber.maximum
  • IJsonSchema.INumber.exclusiveMinimum
  • IJsonSchema.INumber.exclusiveMaximum
  • IJsonSchema.INumber.multipleOf
  • IJsonSchema.IString.minLength
  • IJsonSchema.IString.maxLength
  • IJsonSchema.IString.format
  • IJsonSchema.IString.pattern
  • IJsonSchema.IString.contentMediaType
  • IJsonSchema.IArray.minItems
  • IJsonSchema.IArray.maxItems
  • IJsonSchema.IArray.uniqueItems

By the way, IJsonSchema.IString.format like properties, they sometimes become very important information for the LLM function calling. For example, above id is a string typed property with format: "uuid" constraint. However, OpenAI does not support the format constraint in its JSON schema model.

So with OpenAI’s JSON schema model, every LLM function calling to the UUID typed property will be failed, because OpenAI does not support it. To avoid this critical problem, @agentica’s dependency libraries @samchon/openapi and typia is filling description property with the format constraint information like below.

However, as OpenAI does not understand the @format uuid description text exactly, it may lead to the #Validation Feedback , which corrects the wrong typed parameter values from the LLM function calling. In other words, such non-standard JSON schema model like OpenAI’s IChatGptSchema, it is the principle reason why you have to use @agentica.

{
  "type": "string",
  "description": "Primary Key.\n\n@format uuid"
}

Gemini Schema

src/gemini.schema.ts
import typia from "typia";
 
type Shape = Circle | Rectangle;
interface Circle {
  type: "circle";
  radius: number;
}
interface Rectangle {
  type: "rectangle";
  width: number;
  height: number;
}
typia.llm.schema<Shape>();
Terminal
unsupported type detected
 
- (Circle | Rectangle)\n  - Gemini model does not support the union type.

Google Gemini has its own JSON schema model different from the standard JSON schema specification.

Gemini’s JSON schema model is based on JSON schema draft-07 (OpenAPI v3.0 is following), but it has banned many types and constraint properties like IJsonSchema.IReference.$ref, IJsonSchema.IOneOf.oneOf and IJsonSchema.IInteger.maximum. In detailly, these specifications are not supported in the Gemini’s JSON schema model IGeminiSchema.

As Gemini has banned $ref, oneOf and anyOf, it is not possible to using recursive and union types in Gemini. By the way, such reference and union types are important in many applications, I do not recommend using Gemini for the LLM function calling.

  • Banned types
    • IJsonSchema.IReference.$ref
    • IJsonSchema.IOneOf.oneOf
    • IJsonSchema.IAnyOf.anyOf
  • Banned constraint properties
    • IJsonSchema.INumber.minimum
    • IJsonSchema.INumber.maximum
    • IJsonSchema.INumber.exclusiveMinimum
    • IJsonSchema.INumber.exclusiveMaximum
    • IJsonSchema.INumber.multipleOf
    • IJsonSchema.IString.minLength
    • IJsonSchema.IString.maxLength
    • IJsonSchema.IString.format
    • IJsonSchema.IString.pattern
    • IJsonSchema.IString.contentMediaType
    • IJsonSchema.IArray.minItems
    • IJsonSchema.IArray.maxItems
    • IJsonSchema.IArray.uniqueItems
src/gemini.parameters.ts
import typia, { tags } from "typia";
 
typia.llm.parameters<{
  /**
   * Primary Key.
   * 
   * Primary Key generated by System.
   */
  id: string & tags.Format<"uuid">;
 
  /**
   * Age of the user.
   * 
   * Age of the user in years.
   */
  age: number & tags.Type<"uint32"> & tags.Maximum<100>;
 
  /**
   * Email address of the user.
   * 
   * Email address written when membership joining.
   */
  email: string & tags.Format<"email">;
}>();

Also, IJsonSchema.IString.format like properties, they sometimes become very important information for the LLM function calling. For example, above id is a string typed property with format: "uuid" constraint. However, Gemini does not support the format constraint in its JSON schema model.

So with Gemini’s JSON schema model, every LLM function calling to the UUID typed property will be failed, because Gemini does not support it. To avoid this critical problem, @agentica’s dependency libraries @samchon/openapi and typia is filling description property with the format constraint information like below.

However, as Gemini does not understand the @format uuid description text exactly, it may lead to the #Validation Feedback , which corrects the wrong typed parameter values from the LLM function calling. In other words, such non-standard JSON schema model like Gemini’s IGeminiSchema, it is the principle reason why you have to use @agentica

{
  "type": "string",
  "description": "Primary Key.\n\n@format uuid"
}

Leaderboard

In this page, we’ve learned how to specify and utilize famous LLM vendors.

However, LLM vendors are not only (OpenAI, Claude, DeepSeek, Gemini, Llama). There are much more Local LLMs. We Wrtn Technologies are preparing a leaderboard for a lot of LLM vendors measuring the performance of LLM function calling.

Here is one of the leaderboard example, which is measuring the LLM function calling performance about OpenAI’s o3-mini model. The number means how many attempts are needed to get the correct parameter values from the LLM function calling. If the value is 1, it means that the function calling was successful in the first attempt, and if the value is 2, it means that the function calling was successful in the second attempt with one time validation feedback .

Wait for our leaderboard release. You can choose the best LLM vendor for your application, effiectively and economically.

NameStatus
ObjectConstraint1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
ObjectFunctionSchema2️⃣2️⃣4️⃣2️⃣2️⃣2️⃣2️⃣2️⃣5️⃣2️⃣
ObjectHierarchical1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣2️⃣1️⃣1️⃣2️⃣
ObjectJsonSchema1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
ObjectSimple1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
ObjectUnionExplicit1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
ObjectUnionImplicit1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
ShoppingCartCommodity1️⃣2️⃣2️⃣3️⃣1️⃣1️⃣4️⃣2️⃣1️⃣2️⃣
ShoppingOrderCreate1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
ShoppingOrderPublish1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣❌1️⃣1️⃣1️⃣
ShoppingSaleDetail1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
ShoppingSalePage1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣1️⃣
Last updated on
Agentic AITypeScript Class