Google Generative AI plugin

The Google AI plugin provides a unified interface to connect with Google’s generative AI models through the Gemini Developer API using API key authentication. The @genkit-ai/google-genai package is a drop-in replacement for the previous @genkit-ai/googleai package.

The plugin supports a wide range of capabilities:

• Language Models: Gemini models for text generation, reasoning, and multimodal tasks
• Embedding Models: Text and multimodal embeddings
• Image Models: Gemini image (Nano Banana) models for generation and editing, and Gemini for image analysis
• Video Models: Gemini Omni for video generation and editing, and Gemini for video understanding
• Speech Models: Polyglot text-to-speech generation

Setup

Installation

npm i --save @genkit-ai/google-genai

Configuration

import { genkit } from 'genkit';

import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [
    googleAI(),
    // Or with an explicit API key:
    // googleAI({ apiKey: 'your-api-key' }),
  ],
});

Authentication

Requires a Gemini API Key, which you can get from Google AI Studio. You can provide this key in several ways:

1. Environment variables: Set GEMINI_API_KEY
2. Plugin configuration: Pass apiKey when initializing the plugin (shown above)
3. Per-request: Override the API key for specific requests in the config:

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Your prompt here',
  config: {
    apiKey: 'different-api-key', // Use a different API key for this request
  },
});

This per-request API key option is useful for routing specific requests to different API keys, such as for multi-tenant applications or cost tracking.

Language Models

You can create models that call the Google Generative AI API. The models support tool calls and some have multi-modal capabilities.

Available Models

Gemini 4 and 3 Series - Latest models with state-of-the-art reasoning and multimodal capabilities:

• gemini-4-argon - Gemini 4 Argon, with LOW / MEDIUM / HIGH thinking levels
• gemini-3.8-flash - Most intelligent Flash model, engineered for complex reasoning, coding, and agentic workflows
• gemini-3.6-flash - Fast, capable Flash model
• gemini-3.1-pro-preview - Preview of Gemini 3.1 Pro, for complex reasoning and problem solving
• gemini-3.5-flash-lite - Fastest, most cost-effective model for high-throughput execution
• gemini-nano-banana-2.1 - Fast and efficient image generation and editing
• gemini-3-pro-image - State-of-the-art image generation and editing for complex visual tasks

Gemma 4 Series - Open models for various use cases:

• gemma-4-31b-it - Large instruction-tuned model
• gemma-4-26b-a4b-it - Efficient Mixture-of-Experts instruction-tuned model (4B active parameters)

Latest Aliases - Auto-updating aliases that point to the most recent versions:

• gemini-pro-latest - Points to the latest Gemini Pro model
• gemini-flash-latest - Points to the latest Gemini Flash model
• gemini-flash-lite-latest - Points to the latest Gemini Flash Lite model

Gemini 2.5 models (available only to projects that already used them) and retiring preview models still work, but no longer show up in autocomplete.

Note

See the Google Generative AI models documentation for a complete list of available models and their capabilities.

Interactions API

The plugin sends requests for the -latest aliases, newer Gemini models, and any model it does not know to the Gemini Interactions API. Older built-in models (such as gemini-3.5-flash, gemini-3.1-pro-preview, the Gemini 2.5 family, and Gemma) stay on generateContent until they are retired. The plugin README lists them.

The backend is meant to be invisible: text, reasoning, media, tool calls, streaming, usage, and finish reasons work the same either way. The differences show up only on models routed to the Interactions API:

• response.raw / response.custom is a GeminiInteraction (exported from the package) with a steps[] timeline instead of a GenerateContentResponse with candidates[]. Grounding citations are annotations on the model_output text content.
• toolConfig, thinkingConfig.thinkingBudget, and candidates > 1 are ignored with a warning. Use thinkingConfig.thinkingLevel instead of a budget.
• System messages must be text-only.
• Conversations are not stored on Google’s servers (store: false) unless you opt in with config: { store: true } or googleAI({ store: true }). With store: true, passing the previous response’s messages continues the stored interaction and sends only the new turns.

import type { GeminiInteraction } from '@genkit-ai/google-genai';

// Grounding citations from the model_output steps.
const raw = response.raw as GeminiInteraction;
const annotations = raw.steps
  ?.flatMap((step) => (step.type === 'model_output' ? step.content : []))
  .flatMap((c) => (c.type === 'text' ? (c.annotations ?? []) : []));

Basic Usage

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
});

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Explain how neural networks learn in simple terms.',
});

console.log(response.text);

Model Configuration

You can provide configuration options to tailor the model’s behavior, such as specifying the serviceTier.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-lite-latest'),
  prompt: 'Explain how neural networks learn in simple terms.',
  config: {
    serviceTier: 'flex', // Can be 'standard', 'flex', or 'priority'
  },
});

Structured Output

import { z } from 'genkit';

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  output: {
    schema: z.object({
      name: z.string(),
      bio: z.string(),
      age: z.number(),
    }),
  },
  prompt: 'Generate a profile for a fictional character',
});

console.log(response.output);

Schema Limitations

The Gemini API relies on a specific subset of the OpenAPI 3.0 standard. When defining Zod schemas for structured output, keep the following limitations in mind:

Supported Features

• Objects & Arrays: Standard object properties and array items.
• Enums: Fully supported (z.enum).
• Nullable: Supported via z.nullable() (mapped to nullable: true).

Critical Limitations

• Unions (z.union): Complex unions are often problematic. The API has specific handling for anyOf but may reject ambiguous or complex oneOf structures. Prefer using a single object with optional fields or distinct tool definitions over complex unions.
• Validation Keywords: Keywords like pattern, minLength, maxLength, minItems, and maxItems are not supported by the Gemini API’s constrained decoding. Including them may result in 400 InvalidArgument errors or them being ignored.
• Recursion: Recursive schemas are generally not supported.
• Complexity: Deeply nested schemas or schemas with hundreds of properties may trigger complexity limits.

Best Practices

• Keep schemas simple and flat where possible.
• Use property descriptions (.describe()) to guide the model instead of complex validation rules (e.g., “String must be an email” instead of a regex pattern).
• If you need strict validation (e.g., regex), perform it in your application code after receiving the structured response.

Thinking and Reasoning

Gemini 2.5 and newer models (as well as Gemma 4) use an internal thinking process that improves reasoning for complex tasks.

Thinking Level (Gemini 3.0+ and Gemma 4):

const response = await ai.generate({
  model: googleAI.model('gemini-3.1-pro-preview'),
  prompt: 'what is heavier, one kilo of steel or one kilo of feathers',
  config: {
    thinkingConfig: {
      thinkingLevel: 'HIGH', // Or 'LOW' or 'MEDIUM'
      includeThoughts: true, // Include thought summaries
    },
  },
});

if (response.reasoning) {
  console.log('Reasoning:', response.reasoning);
}

Thinking Budget (Gemini 2.5): thinkingConfig.thinkingBudget is sent only to models on generateContent. It was designed for the legacy Gemini 2.5 models, which are available only to projects that already used them. Models routed to the Interactions API ignore it with a warning; use thinkingLevel instead.

Context Caching

Gemini 2.5 and newer models automatically cache common content prefixes (min 2,048 tokens for Gemini 2.5, 4,096 for Gemini 3.1 Pro Preview, Gemini 3.6 and 3.8 Flash, and Gemini 4 Argon). Cached tokens are billed at a fraction of the input price; see pricing.

// Structure prompts with consistent content at the beginning
const baseContext = `You are a helpful cook... (large context) ...`.repeat(50);

// First request - content will be cached
await ai.generate({
  model: googleAI.model('gemini-pro-latest'),
  prompt: `${baseContext}\n\nTask 1...`,
});

// Second request with same prefix - eligible for cache hit
await ai.generate({
  model: googleAI.model('gemini-pro-latest'),
  prompt: `${baseContext}\n\nTask 2...`,
});

Safety Settings

You can configure safety settings to control content filtering for different harm categories.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Your prompt here',
  config: {
    safetySettings: [
      {
        category: 'HARM_CATEGORY_HATE_SPEECH',
        threshold: 'BLOCK_MEDIUM_AND_ABOVE',
      },
      {
        category: 'HARM_CATEGORY_DANGEROUS_CONTENT',
        threshold: 'BLOCK_MEDIUM_AND_ABOVE',
      },
    ],
  },
});

Available harm categories:

• HARM_CATEGORY_HATE_SPEECH
• HARM_CATEGORY_DANGEROUS_CONTENT
• HARM_CATEGORY_HARASSMENT
• HARM_CATEGORY_SEXUALLY_EXPLICIT

Available thresholds:

• BLOCK_LOW_AND_ABOVE
• BLOCK_MEDIUM_AND_ABOVE
• BLOCK_ONLY_HIGH
• BLOCK_NONE

Accessing Safety Ratings:

Safety ratings are typically only included when content is flagged. On generateContent models, you can access them from the response custom metadata:

const geminiResponse = response.custom as any;
const candidateSafetyRatings = geminiResponse?.candidates?.[0]?.safetyRatings;
const promptSafetyRatings = geminiResponse?.promptFeedback?.safetyRatings;

Deep Research

Deep Research models can perform extensive research tasks over multiple turns, using specialized workflows.

Available Models:

• deep-research-preview-04-2026
• deep-research-max-preview-04-2026

deep-research-pro-preview-12-2025 was shut down on October 23, 2026.

Usage:

let { operation } = await ai.generate({
  model: googleAI.model('deep-research-preview-04-2026'),
  prompt: 'Analyze global semiconductor market trends. Include graphics showing market share changes.',
  config: {
    visualization: 'AUTO',
  },
});

if (!operation) throw new Error('No operation returned');

// Deep research operations are long-running and need to be polled
while (!operation.done) {
  operation = await ai.checkOperation(operation);
  await new Promise((resolve) => setTimeout(resolve, 30000)); // Check every 30 seconds
}

console.log(operation.output?.message?.content);

You can also use previousInteractionId for multi-turn research, set collaborativePlanning: true to get a research plan first, or use ai.cancelOperation(operation) to halt an ongoing research task.

Antigravity

The Antigravity agent works in a sandbox environment that persists across turns:

const response = await ai.generate({
  model: googleAI.model('antigravity-preview-09-2026'),
  prompt: 'Read Hacker News, summarize the top 10 stories.',
  // The sandbox the agent works in ('remote' provisions a fresh one).
  config: { environment: 'remote' },
});

To continue, pass the previous response’s messages (plus your next message) in a follow-up call; the plugin continues the stored interaction in the same environment. Antigravity continues turns through stored interactions (its function calling and background execution also require them), so unlike Gemini models it doesn’t default to store: false. Don’t set store: false for it; if you use googleAI({ store: false }), set store: true in the Antigravity request config.

Google Search Grounding

Enable Google Search to provide answers with current information and verifiable sources.

import type { GeminiInteraction } from '@genkit-ai/google-genai';

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'What are the top tech news stories this week?',
  config: {
    googleSearchRetrieval: true,
  },
});

// Sources: URL citations on the model_output text.
const raw = response.raw as GeminiInteraction;
const sources = raw.steps
  ?.flatMap((step) => (step.type === 'model_output' ? step.content : []))
  .flatMap((c) => (c.type === 'text' ? (c.annotations ?? []) : []))
  .filter((a) => a.type === 'url_citation');
console.log('Sources:', sources);

Models on generateContent (such as gemini-3.1-pro-preview) return grounding metadata on the first candidate instead:

const groundingMetadata = (response.custom as any)?.candidates?.[0]
  ?.groundingMetadata;
if (groundingMetadata) {
  console.log('Sources:', groundingMetadata.groundingChunks);
}

The following configuration options are available for Google Search grounding:

• googleSearchRetrieval object | boolean

  Enables Google Search grounding. Can be a boolean (true) or a configuration object. Example: { dynamicRetrievalConfig: { mode: 'MODE_DYNAMIC', dynamicThreshold: 0.7 } }

  • dynamicRetrievalConfig object
    • mode string The retrieval mode (e.g., 'MODE_DYNAMIC').
    • dynamicThreshold number The threshold for dynamic retrieval (e.g., 0.7).

Response Metadata (generateContent models):

• webSearchQueries string[]

  Array of search queries used to retrieve information. Example: ["What's the weather in Chicago this weekend?"]

• searchEntryPoint object

  Contains the main search result content formatted for display.

  • renderedContent string The HTML content of the search result.

• groundingSupports object[]

  Links specific response segments to supporting search result chunks.

  • segment object
    • text string The text of the segment.
  • groundingChunkIndices number[] Indices of the chunks that support this segment.
  • confidenceScores number[] Confidence scores for each supporting chunk.

File Search Grounding

Ground the model’s responses using documents stored in Google’s File Search API.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: "What is the character's name in the story?",
  config: {
    fileSearch: {
      fileSearchStoreNames: ['fileSearchStores/my-store-123'],
      metadataFilter: 'author=foo',
    },
  },
});

Google Maps Grounding

Enable Google Maps to provide location-aware responses.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Find coffee shops near the CN Tower',
  config: {
    tools: [{ googleMaps: {} }],
  },
});

You can also request a widget token to render an interactive map:

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Show me a map of San Francisco',
  config: {
    tools: [{ googleMaps: { enableWidget: true } }],
  },
});

The following configuration options are available for Google Maps grounding:

• googleMaps object

  Enables Google Maps grounding. Example: { enableWidget: true }

  • enableWidget boolean Whether to include a widget token in the response.

• retrievalConfig object

  Additional configuration for provider tools. Can improve relevance by providing location context for Google Maps. Example: { retrievalConfig: { latLng: { latitude: 37.7749, longitude: -122.4194 } } }

  • retrievalConfig object
    • latLng object
      • latitude number The latitude in degrees.
      • longitude number The longitude in degrees.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Describe some sights near me',
  config: {
    tools: [{ googleMaps: {} }],
    retrievalConfig: {
      latLng: {
        latitude: 43.0896,
        longitude: -79.0849,
      },
    },
  },
});

URL Context

Provide specific URLs for the model to analyze:

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Summarize this page',
  config: {
    tools: [{ urlContext: {} }],
  },
});

When using urlContext, the model will fetch content from URLs found in your prompt.

Combine built-in tools and Genkit tools

You can combine Gemini built-in tools and Genkit tools defined using ai.defineTool. Built-in tools are specified in the tools property of the config object, while Genkit tools are provided in the top-level tools property.

// const getWeather = ai.defineTool(...);

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt:
    'What is the southernmost city in Canada? What is the weather like there today?',
  config: {
    tools: [{ googleSearch: {} }], // Built-in tools are defined in the config
  },
  tools: [getWeather], // Genkit tools are defined in top-level tools
});

Models routed to the Interactions API always include server-side tool invocations, so they don’t need toolConfig (and ignore it with a warning). On generateContent models, set toolConfig: { includeServerSideToolInvocations: true } in config.

Code Execution

Enable the model to write and execute Python code for calculations and logic.

const response = await ai.generate({
  model: googleAI.model('gemini-pro-latest'),
  prompt: 'Calculate the 20th Fibonacci number',
  config: {
    codeExecution: true,
  },
});

The following configuration options are available for code execution:

• codeExecution boolean

  Enables code execution for reasoning and calculations. Example: true

Generating Text and Images (Nano Banana)

Some Gemini models (like gemini-nano-banana-2.1, gemini-3-pro-image) can output images natively alongside text:

const response = await ai.generate({
  model: googleAI.model('gemini-nano-banana-2.1'),
  prompt: 'Create a picture of a futuristic city and describe it',
  config: {
    responseModalities: ['IMAGE', 'TEXT'],
  },
});

// Extract image
if (response.media) {
  console.log('Image:', response.media.url);
}

// Extract text
if (response.text) {
  console.log('Text:', response.text);
}

// Extract all messages including text and images
if (response.messages) {
  console.log('Messages:', response.messages);
}

The following configuration options are available for Gemini image generation:

• responseModalities string[]

  Specifies the output modalities. Options: ['TEXT', 'IMAGE'], ['IMAGE'] Default: ['TEXT', 'IMAGE']

• imageConfig object

  • aspectRatio string

    Aspect ratio of the generated images. Not all models support all aspect ratios. Options: '1:1', '1:4', '1:8', '2:3', '3:2', '3:4', '4:1', '4:3', '4:5', '5:4', '8:1', '9:16', '16:9', '21:9' Default: '1:1'

  • imageSize string

    Resolution of the generated image. Supported by Gemini 3+ image models only. Options: '1K', '2K', '4K' Default: '1K'

Multimodal Input Capabilities

Video Understanding

Gemini models can process videos to describe content, answer questions, and refer to timestamps (in MM:SS format).

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: [
    { text: 'What happens at 00:05?' },
    {
      media: {
        contentType: 'video/mp4',
        url: 'https://youtube.com/watch?v=...',
      },
    },
  ],
});

Video Processing Details:

• Sampling: 1 frame per second (default)
• Context: 2M context models can handle up to 2 hours of video.
• Inputs: Up to 10 videos per request (Gemini 2.5+).

Agentic processing: Gemini 4 Argon, 3.8 Flash, 3.6 Flash, and 3.5 Flash-Lite, which the plugin routes to the Interactions API, can explore a video and load only the parts they need instead of reading frames at a fixed rate, which works better for long videos. Set metadata: { processing: 'agentic' } on the video part (the default is 'static'):

const response = await ai.generate({
  model: googleAI.model('gemini-4-argon'),
  prompt: [
    {
      media: { url: 'https://www.youtube.com/watch?v=...' },
      metadata: { processing: 'agentic' },
    },
    {
      text: 'What happens in this video? List the main moments with timestamps.',
    },
  ],
});

The model’s loading steps come back as custom.processingCall and custom.processingResult parts; keep them in the history for follow-up questions. response.usage.custom.toolUseTokens counts the tokens loaded.

Image Understanding

Gemini models can reason about images passed as inline data or URLs.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: [
    { text: 'Describe what is in this image' },
    { media: { url: 'https://example.com/image.jpg' } },
  ],
});

Audio Understanding

Gemini models can process audio files to transcribe speech text, answer questions about the audio content, or summarize recordings.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: [
    { text: 'Transcribe this audio clip' },
    {
      media: { contentType: 'audio/mp3', url: 'https://example.com/audio.mp3' },
    },
  ],
});

PDF Support

Gemini models can process PDF documents to extract information, summarize content, or answer questions based on the visual layout and text.

const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: [
    { text: 'Summarize this document' },
    {
      media: {
        contentType: 'application/pdf',
        url: 'https://example.com/doc.pdf',
      },
    },
  ],
});

File Inputs and Gemini Files API

Gemini models support various file types. For small files, you can use inline data. For larger files (up to 2GB), use the Gemini Files API.

Using Files API:

To use large files, you must upload them using the Google GenAI SDK or other supported methods. Genkit does not provide file management helpers, but you can pass the file URI to Genkit for generation:

import { GoogleGenAI } from '@google/genai';
// ... init genaiClient ...

// Upload file
const uploadedFile = await genaiClient.files.upload({
  file: 'path/to/video.mp4',
  config: { mimeType: 'video/mp4' },
});

// Use in generation
const response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: [
    { text: 'Describe this video' },
    {
      media: {
        contentType: uploadedFile.mimeType,
        url: uploadedFile.uri,
      },
    },
  ],
});

Embedding Models

Available Models

• gemini-embedding-2 — Latest stable embedding model with 3072 dimensions; supports multimodal input (text, images, video).
• gemini-embedding-001 — Default 3072 dimensions; set outputDimensionality in embed params (for example 768, 1536, or 3072) when you want a shorter vector.

gemini-embedding-2-preview is no longer in the plugin’s model list; use gemini-embedding-2.

Usage

const embeddings = await ai.embed({
  embedder: googleAI.embedder('gemini-embedding-001'),
  content: 'Machine learning models process data to make predictions.',
});

console.log(embeddings);

// Optional: request a shorter embedding (size indexes to match)
const compact = await ai.embed({
  embedder: googleAI.embedder('gemini-embedding-001'),
  content: 'Machine learning models process data to make predictions.',
  options: { outputDimensionality: 768 },
});

Image Models

Available Models

• gemini-nano-banana-2.1 - Fast and efficient image generation and editing
• gemini-3-pro-image - State-of-the-art image generation and editing for complex visual tasks

The Imagen models on the Gemini API have been shut down, and imagen-* model names fail with an error that points to gemini-nano-banana-2.1. Imagen on Vertex AI was also retired, on June 30, 2026, so use the Gemini image models there too; see the Vertex AI plugin.

Usage

const response = await ai.generate({
  model: googleAI.model('gemini-nano-banana-2.1'),
  prompt: 'A serene Japanese garden with cherry blossoms and a koi pond.',
  config: {
    imageConfig: { aspectRatio: '16:9' },
  },
});

const generatedImage = response.media;

Image settings go in config.imageConfig; the type is exported as GeminiImageConfig (ImagenConfig is no longer exported). See Generating Text and Images (Nano Banana) for the options.

Video Models

The Google AI plugin generates video with Gemini Omni. It takes text, images, and videos as input, so the same model covers text to video, image to video, and editing or extending a video.

Available Models

• gemini-omni-1.1-flash - Video generation and editing

Gemini Omni replaces the Veo 3.1 preview models on the Gemini API, which were shut down on October 22, 2026. Veo on Vertex AI is not affected; see the Vertex AI plugin.

Usage

Text-to-Video

Unlike Veo, Gemini Omni is not a long-running operation: ai.generate() returns the video directly.

import { googleAI } from '@genkit-ai/google-genai';
import { genkit } from 'genkit';

const ai = genkit({
  plugins: [googleAI()],
});

const response = await ai.generate({
  model: googleAI.model('gemini-omni-1.1-flash'),
  prompt: 'An origami butterfly flaps its wings and flies out of the french doors into the garden.',
  config: {
    videoFormat: { aspectRatio: '16:9', resolution: '720p' },
  },
});

// An MP4 video (video/mp4), returned inline as a data URL.
const video = response.media;

Video from a photo reference

Images and videos in the prompt are used as inputs, for example to make a static photo move:

import * as fs from 'fs';

const startingImage = fs.readFileSync('photo.jpg', { encoding: 'base64' });

const response = await ai.generate({
  model: googleAI.model('gemini-omni-1.1-flash'),
  prompt: [
    { text: 'Make the subject in the photo move.' },
    { media: { contentType: 'image/jpeg', url: `data:image/jpeg;base64,${startingImage}` } },
  ],
  config: {
    videoFormat: { aspectRatio: '9:16' },
  },
});

Edit a generated video

To edit a generated video in a follow-up turn, set store: true and pass the previous response’s messages. The plugin continues the stored interaction:

const first = await ai.generate({
  model: googleAI.model('gemini-omni-1.1-flash'),
  prompt: 'A paper boat drifting down a rainy street.',
  config: { store: true },
});

const edited = await ai.generate({
  model: googleAI.model('gemini-omni-1.1-flash'),
  messages: [
    ...first.messages,
    { role: 'user', content: [{ text: 'Make it sunset instead of rain.' }] },
  ],
  config: { store: true },
});

The following configuration options are available for video generation (GeminiVideoConfig):

• videoFormat object

  • aspectRatio string: '16:9' (default) or '9:16'.
  • resolution string: '360p', '720p' (default), '1080p', or '4k'. 1080p and 4k are upscaled.
  • delivery string: 'uri' returns a Files API URI instead of inline data, which the Gemini API recommends for videos larger than 4 MB (usually above 720p). It requires store: true. Download the file with your API key in the x-goog-api-key header. It can take a few seconds after the response to become ACTIVE, and it expires after 48 hours.

• videoConfig object

  • task string: 'text_to_video', 'image_to_video', 'reference_to_video', 'edit', or 'extend'. Tells the model which kind of generation to do; without it, the model infers it from the prompt.

Tools are not supported on video models. Gemini Omni also doesn’t support system instructions, temperature, topP, or stop sequences.

Music Models (Lyria)

The Google AI plugin provides access to music and audio generation capabilities through the Lyria models.

Available Models

• lyria-3.5 - Latest music generation model
• lyria-3-pro-preview - High-quality music generation
• lyria-3-clip-preview - Fast generation for short music clips

Usage

const response = await ai.generate({
  model: googleAI.model('lyria-3.5'),
  prompt: 'A cheerful acoustic folk song with guitar and harmonica.',
});

// Access the generated audio media
const audioMedia = response.media;

Speech Models

The Google GenAI plugin provides access to text-to-speech capabilities through Gemini TTS models. These models can convert text into natural-sounding speech for various applications.

Available Models

• gemini-3.8-flash-tts - Gemini 3.8 Flash model with TTS
• gemini-3.8-flash-lite-tts - Gemini 3.8 Flash Lite model with TTS

The 2.5 and 3.1 preview TTS models (gemini-2.5-flash-preview-tts, gemini-2.5-pro-preview-tts, gemini-3.1-flash-tts-preview) are deprecated and will shut down on November 17, 2026.

Usage

Basic Usage

Pass a speechConfig object with voiceConfig set. You’ll need to choose a voice name from the prebuilt output voices. Delivery instructions go in the part’s metadata, since the text itself is read verbatim:

const response = await ai.generate({
  model: googleAI.model('gemini-3.8-flash-tts'),
  prompt: [
    {
      text: 'Have a wonderful day!',
      metadata: { speechMetadata: { style: 'cheerful and friendly' } },
    },
  ],
  config: {
    speechConfig: {
      voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Kore' } },
    },
  },
});

// A complete WAV file (audio/wav), ready to play or save.
const audio = response.media;

Audio format and streaming

• Without audioFormat, a non-streamed request returns one audio/wav file, and a streamed request returns raw PCM (audio/l16, 24 kHz, mono, 16-bit, no header).
• Set config.audioFormat.mimeType to choose the format: audio/wav, audio/l16, audio/mulaw, or audio/alaw.
• When streaming, audio chunks arrive as the speech is generated. The final response joins them into one media part in the requested format, so request audio/wav when streaming to get a playable file at the end.

const { stream, response } = ai.generateStream({
  model: googleAI.model('gemini-3.8-flash-lite-tts'),
  prompt: 'Have a wonderful day!',
  config: {
    speechConfig: {
      voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Kore' } },
    },
    audioFormat: { mimeType: 'audio/wav' },
  },
});

for await (const chunk of stream) {
  // chunk.media: the next piece of audio.
}

const audio = (await response).media; // One complete WAV file.

Raw PCM output

The deprecated preview TTS models return raw PCM data, which you can convert to a standard format like WAV:

import wav from 'wav';
import { Buffer } from 'node:buffer';

async function saveWavFile(
  filename: string,
  pcmData: Buffer,
  sampleRate = 24000,
) {
  return new Promise((resolve, reject) => {
    const writer = new wav.FileWriter(filename, {
      channels: 1,
      sampleRate,
      bitDepth: 16,
    });
    writer.on('finish', resolve);
    writer.on('error', reject);
    writer.write(pcmData);
    writer.end();
  });
}

const response = await ai.generate({
  model: googleAI.model('gemini-3.1-flash-tts-preview'),
  config: {
    responseModalities: ['AUDIO'],
    speechConfig: {
      voiceConfig: {
        prebuiltVoiceConfig: { voiceName: 'Algenib' },
      },
    },
  },
  prompt: 'Say that Genkit is an amazing AI framework',
});

if (response.media?.url) {
  const data = response.media.url.split(',')[1];
  if (data) {
    const pcmData = Buffer.from(data, 'base64');
    await saveWavFile('output.wav', pcmData);
  }
}

Multi-Speaker

You can generate audio with multiple speakers, each with their own voice. Put each turn in its own text part and name its speaker in the part’s metadata, matching a configured speaker:

const { media } = await ai.generate({
  model: googleAI.model('gemini-3.8-flash-lite-tts'),
  prompt: [
    {
      text: "How's it going today, Jane?",
      metadata: { speechMetadata: { speaker: 'Joe' } },
    },
    {
      text: 'Not too bad, how about you?',
      metadata: { speechMetadata: { speaker: 'Jane', style: 'calm' } },
    },
  ],
  config: {
    speechConfig: {
      multiSpeakerVoiceConfig: {
        speakerVoiceConfigs: [
          {
            speaker: 'Joe',
            voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Puck' } },
          },
          {
            speaker: 'Jane',
            voiceConfig: { prebuiltVoiceConfig: { voiceName: 'Kore' } },
          },
        ],
      },
    },
  },
});

The deprecated preview TTS models instead detect speaker labels in a single text (like “Speaker A:” and “Speaker B:”).

The following configuration options are available for speech generation:

• speechConfig object

  • voiceConfig object

    Defines the voice configuration for a single speaker.

    • prebuiltVoiceConfig object

      • voiceName string

        The name of the voice to use. Options: Puck, Charon, Kore, Fenrir, Aoede (and others).

  • multiSpeakerVoiceConfig object

    Defines the voice configuration for multiple speakers.

    • speakerVoiceConfigs array

      A list of voice configurations for each speaker.

      • speaker string

        The name of the speaker. On the 3.8 TTS models, it matches metadata.speechMetadata.speaker on each text part; the deprecated preview models match it against labels in the prompt (e.g., “Speaker A”).

      • voiceConfig object

        The voice configuration for this speaker. See voiceConfig above.

Speech Emphasis

With the deprecated preview TTS models, you can use markdown-style formatting in your prompt to add emphasis:

• Bold text (**like this**) for stronger emphasis.
• Italic text (*like this*) for moderate emphasis.

prompt: 'Genkit is an **amazing** Gen AI *library*!';

The 3.8 TTS models read the text verbatim, so set the delivery, including pace, pitch, and volume, with metadata.speechMetadata.style instead, as in the basic usage example above, and capitalize a word (like VERY) to stress it.

TTS models automatically detect the input language. Supported languages include en-US, fr-FR, de-DE, es-US, ja-JP, ko-KR, pt-BR, zh-CN, and more.