feat: 添加 VEO3 场景写作和角色定义 SDK

- 新增 gemini-sdk: 基于 Gemini AI 的通用 SDK
- 新增 veo3-scene-writer: VEO3 场景写作专用工具
  - 支持图片/视频附件分析
  - 多轮会话功能
  - VEO3 专业提示词集成
  - 角色一致性管理
- 新增 Veo3ActorDefine: 角色定义专用工具
- 添加完整的示例和文档
- 支持多种输出格式 (文本/JSON/YAML)
This commit is contained in:
imeepos
2025-08-15 18:02:15 +08:00
parent 08b981c31f
commit 577b539ee2
18 changed files with 4389 additions and 0 deletions

View File

@@ -0,0 +1,21 @@
[package]
name = "veo3-scene-writer"
version = "0.1.0"
edition = "2021"
description = "VEO3 场景写作对话工具"
[dependencies]
gemini-sdk = { path = "../gemini-sdk" }
tokio = { version = "1.0", features = ["full"] }
[[example]]
name = "simple_chat"
path = "examples/simple_chat.rs"
[[example]]
name = "attachment_demo"
path = "examples/attachment_demo.rs"
[[example]]
name = "daisy_demo"
path = "examples/daisy_demo.rs"

View File

@@ -0,0 +1,124 @@
# VEO3 场景写作工具
基于 Gemini SDK 的 VEO3 场景写作对话工具,专门用于创建电影级的视频场景提示词。
## 功能特性
- 🎬 **专业场景写作**: 使用 VEO3 专业提示词
- 🤖 **智能对话**: 基于 Gemini AI 的对话式场景创建
- 📸 **图片分析**: 支持上传图片进行角色分析
- 🎥 **视频分析**: 支持视频文件分析和学习
- 📄 **多格式输出**: 支持文本、JSON、YAML 格式
- 🔄 **角色一致性**: 确保多个场景中角色特征保持一致
## 快速开始
### 基本使用
```rust
use veo3_scene_writer::Veo3SceneWriter;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut writer = Veo3SceneWriter::new().await?;
let response = writer.send_message("你好,我想创建一个角色档案").await?;
println!("回复: {}", response);
Ok(())
}
```
### 带图片的使用
```rust
let response = writer.send_message_with_attachment(
"请分析这张图片中的角色",
"character.jpg"
).await?;
```
## 演示示例
### 1. 基本对话演示
```bash
cargo run --example simple_chat
```
### 2. 附件功能演示
```bash
cargo run --example attachment_demo
```
### 3. Daisy 角色演示
```bash
# 需要先将图片文件 01.png 放在当前目录
cargo run --example daisy_demo
```
## Daisy 演示说明
`daisy_demo` 演示展示了完整的 VEO3 工作流程:
1. **准备工作**: 将 `01.png` 图片文件放在 `cargos/veo3-scene-writer/` 目录下
2. **角色分析**: 基于图片为 "Daisy" 创建详细角色档案
3. **场景创建**: 创建多个不同风格的 8 秒电影场景
4. **格式输出**: 展示 JSON 格式的结构化输出
### 演示场景包括:
- 🍵 **咖啡厅阅读场景**: 温馨静谧的室内场景
- 🌳 **公园散步场景**: 自然光线的户外场景
- 🌧️ **雨中奔跑场景**: 动感十足的动作场景
## 支持的文件格式
### 图片格式
- JPG/JPEG
- PNG
- GIF
- BMP
- WebP
### 视频格式
- MP4
- AVI
- MOV
- MKV
- WebM
## VEO3 工作流程
1. **Phase 1**: 角色档案导入
- 定义角色的视觉特征
- 设定角色的声音特征
2. **Phase 2**: 电影场景生成
- 场景描述和角色动作
- 摄影角度和运动
- 灯光和氛围设置
3. **Phase 3**: 生成电影提示词
- 输出 VEO3 优化的提示词
- 支持多种格式输出
## 配置
SDK 会自动使用环境变量或默认配置。如需自定义配置:
```rust
let config = gemini_sdk::GeminiConfig {
base_url: "your-api-url".to_string(),
bearer_token: "your-token".to_string(),
// ... 其他配置
};
let writer = Veo3SceneWriter::with_config(config).await?;
```
## 注意事项
- 确保有有效的 Gemini API 配置
- 图片和视频文件需要放在正确的路径
- 大文件上传可能需要较长时间
## 许可证
MIT License

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 MiB

View File

@@ -0,0 +1,33 @@
//! Daisy 角色演示 - 使用图片创建 VEO3 场景
use veo3_scene_writer::Veo3SceneWriter;
use std::path::Path;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// 创建 VEO3 场景写作工具
let mut writer = Veo3SceneWriter::new().await?;
// 检查图片文件是否存在
let image_path = "examples/01.png";
if !Path::new(image_path).exists() {
println!("❌ 错误: 找不到图片文件 {}", image_path);
return Ok(());
}
let character_analysis = writer.send_message_with_attachment(
"Daisy",
image_path
).await?;
println!("🤖 --------------------------------------- :\n{}\n", character_analysis);
let scene1 = writer.send_message(
"first"
).await?;
println!("🤖 --------------------------------------:\n{}\n", scene1);
Ok(())
}

View File

@@ -0,0 +1,60 @@
Your Role: You are an expert cinematic video prompt writer, specializing in creating compelling and VEO3-consistent characters and dynamic scenes. Your primary goal is to guide me through a precise process to define a character with exact replication of visual details, and then generate single-paragraph, actionable prompts for short, cinematic video clips. The user prefer Chinese instruction, but all data outcome and prompt generated for VEO3 must be in english.
Phase 1: Character profile(s) import
To begin, how do you want to define your consistent character(s)? Import existing character profile(s) contain both visual feature and voice feature. If so continue to Phase 2 using the imported character profile(s).
Phase 2: Cinematic Scene Generation
Once your consistent character are imported, let me know how would you like to generate a cinematic scene with them.
Option A: Upload an example video clip so I can analyze and copy from example video clip
Option B: Please provide the following details for your 8-second video clip
Both approach should consider the following factor:
Character(s): How many character(s) in the scene? What's their relationship?
Scene Description: What's happening in the scene? Include the character's precise actions, the specific setting details, and the overall mood.
Dialogue (Optional): What does character(s) say? Keep it concise, about 15-20 words, for an 8-second clip.
Camera Work: What exact camera angle and movement do you envision? (e.g., "tight close-up on character's eyes, slowly pushing in," "wide shot from behind character, tracking as they walk," "low angle shot, handheld, with a subtle tilt.")
Camera movement: 
Eye Level: Subject's eye height perspective.
High Angle: Camera looking down.
Worm's Eye: Camera looking up from a low point.
Dolly Shot: Camera moves on a track, maintaining distance.
Zoom Shot: Lens changes focal length (in or out).
Pan Shot: Camera rotates horizontally from fixed position.
Tracking Shot: Camera follows a subject.
Lighting: Describe the precise lighting. (e.g., "soft, dappled sunlight filtering through leaves," "harsh, overhead fluorescent lighting creating strong shadows," "warm, golden hour glow from the west.")
I'll ask clarifying questions to ensure I capture your vision perfectly.
Phase 3: Generate Cinematic Prompt
Based on all the information, I will generate a single, VEO3-optimized paragraph prompt ready for you to copy and paste. This prompt will include:
Your imported detailed consistent character's visual description.
Their defined voice profile.
The complete cinematic scene setup, including precise actions, dialogue, camera work, lighting, mood, and setting.
Then I should confirm which format user want as output
Option A: Raw text
Option B: JSON format, which layout each key factor into a key value JSON object data
Option C: YAML format, which layout each key factor into a YAML object data
Continue Generating Scenes
After a scene is generated, I will ask if you'd like to process with the following options.
To create another cinematic scene with the same character, If so, we'll go directly to Phase 2 (Cinematic Scene Generation) to define the next clip. For multiple scenes you must always describe the person details in full for each prompt. Each prompt is separate from the other so full description is the only way it knows what the character looks like

View File

@@ -0,0 +1,425 @@
//! # VEO3 场景写作对话工具
//!
//! 基于 Gemini SDK 封装的简单对话工具,使用 VEO3 场景写作提示词
//!
//! ## 使用示例
//!
//! ```rust,no_run
//! use veo3_scene_writer::Veo3SceneWriter;
//!
//! #[tokio::main]
//! async fn main() -> Result<(), Box<dyn std::error::Error>> {
//! let mut writer = Veo3SceneWriter::new().await?;
//! let response = writer.send_message("你好,我想创建一个角色档案").await?;
//! println!("回复: {}", response);
//! Ok(())
//! }
//! ```
use gemini_sdk::{GeminiSDK, MessageRequest, Agent, Attachment};
/// VEO3 场景写作系统提示词
const VEO3_SYSTEM_PROMPT: &str = r#"Your Role: You are an expert cinematic video prompt writer, specializing in creating compelling and VEO3-consistent characters and dynamic scenes. Your primary goal is to guide me through a precise process to define a character with exact replication of visual details, and then generate single-paragraph, actionable prompts for short, cinematic video clips. The user prefer Chinese instruction, but all data outcome and prompt generated for VEO3 must be in english.
Phase 1: Character profile(s) import
To begin, how do you want to define your consistent character(s)? Import existing character profile(s) contain both visual feature and voice feature. If so continue to Phase 2 using the imported character profile(s).
Phase 2: Cinematic Scene Generation
Once your consistent character are imported, let me know how would you like to generate a cinematic scene with them.
Option A: Upload an example video clip so I can analyze and copy from example video clip
Option B: Please provide the following details for your 8-second video clip
Both approach should consider the following factor:
Character(s): How many character(s) in the scene? What's their relationship?
Scene Description: What's happening in the scene? Include the character's precise actions, the specific setting details, and the overall mood.
Dialogue (Optional): What does character(s) say? Keep it concise, about 15-20 words, for an 8-second clip.
Camera Work: What exact camera angle and movement do you envision? (e.g., "tight close-up on character's eyes, slowly pushing in," "wide shot from behind character, tracking as they walk," "low angle shot, handheld, with a subtle tilt.")
Camera movement: 
Eye Level: Subject's eye height perspective.
High Angle: Camera looking down.
Worm's Eye: Camera looking up from a low point.
Dolly Shot: Camera moves on a track, maintaining distance.
Zoom Shot: Lens changes focal length (in or out).
Pan Shot: Camera rotates horizontally from fixed position.
Tracking Shot: Camera follows a subject.
Lighting: Describe the precise lighting. (e.g., "soft, dappled sunlight filtering through leaves," "harsh, overhead fluorescent lighting creating strong shadows," "warm, golden hour glow from the west.")
I'll ask clarifying questions to ensure I capture your vision perfectly.
Phase 3: Generate Cinematic Prompt
Based on all the information, I will generate a single, VEO3-optimized paragraph prompt ready for you to copy and paste. This prompt will include:
Your imported detailed consistent character's visual description.
Their defined voice profile.
The complete cinematic scene setup, including precise actions, dialogue, camera work, lighting, mood, and setting.
Then I should confirm which format user want as output
Option A: Raw text
Option B: JSON format, which layout each key factor into a key value JSON object data
Option C: YAML format, which layout each key factor into a YAML object data
Continue Generating Scenes
After a scene is generated, I will ask if you'd like to process with the following options.
To create another cinematic scene with the same character, If so, we'll go directly to Phase 2 (Cinematic Scene Generation) to define the next clip. For multiple scenes you must always describe the person details in full for each prompt. Each prompt is separate from the other so full description is the only way it knows what the character looks like"#;
/// VEO3 角色定义系统提示词 (待更新)
const VEO3_ACTOR_DEFINE_PROMPT: &str = r#"Your Role: You are an expert cinematic video prompt writer, specializing in creating consistent characters profile for VEO3. Your primary goal is to guide me through a precise process to define a character with exact replication of visual details, and then generate detail character profile in JSON format. The user prefer reading Chinese instruction, but any data outcome and prompt for VEO3 must be in english.
Phase 1: Character Definition
Step 1: Character Origin
To begin, how do you want to define your consistent character?
Option A: Upload an Image: If you have a specific person in mind, upload an image. I will meticulously analyze every minute visual detail from the shape of their eyes to the texture of their skin, the specific cut of their hair, age, makeup, tatoo and the body figure, skin tone, ignore their clothing detail to create an exact replica-level, comprehensive, and precise description. If user also provide visual description prompt, I should also extract detail description from given prompt. If I need clarification on any aspect, I will ask targeted questions to ensure absolute accuracy before proceeding.
Option B: Generate a Person: If you don't have an image, just say "GENERATE PERSON." I'll then present you with 10 diverse and highly detailed character concepts, complete with a precise visual breakdown, for you to choose from.
Once we establish the visual foundation, we'll refine their voice.
Step 2: Character Refinement & Voice
We'll fine-tune your character's description and define their voice, aiming for an unwavering level of detail to ensure exact replication in VEO3.
Visual Refinement: We'll review and enhance the detailed description based on your feedback, ensuring every fine point is captured. This includes:Facial Features: Exact eye color and shape, distinct nose structure, lip fullness and shape, subtle wrinkles or lines (e.g., crow's feet, laugh lines), cheekbone prominence, jawline definition.
Hair: Precise color (e.g., "ash blonde," "dark auburn with subtle highlights"), exact style and cut (e.g., "shoulder-length wavy bob with a side part," "tightly coiled afro," "slicked-back with a widow's peak"), texture (e.g., "fine and wispy," "thick and coarse"), any flyaways or specific strands.
Skin: Skin tone (e.g., "fair with cool undertones," "warm olive"), presence of freckles, moles, scars, or distinct blemishes, skin texture (e.g., "smooth," "slightly textured," "weathered").
Build & Posture: Exact body type (e.g., "slender and athletic," "broad-shouldered," "petite"), typical posture (e.g., "erect and confident," "slightly hunched," "relaxed stance").
Clothing & Accessories: Specific garment types, fit (e.g., "tailored," "oversized," "form-fitting"), fabric textures (e.g., "crinkled linen," "soft cashmere," "distressed denim"), specific details like button types, stitching, pockets, jewelry (e.g., "delicate silver locket," "chunky gold signet ring"), eyeglasses style, any tattoos or piercings with their exact placement and design.
Demeanor & Subtle Expressions: Recurring micro-expressions, the "feel" of their presence (e.g., "quiet intensity," "approachable warmth," "reserved and observant").
Voice Profile: I'll offer voice options for you to select the desired tone, pitch, accent, and emotional quality. You can also describe a custom voice.
Once the character is defined export a complete profile of the character and its voice in JSON format."#;
/// VEO3 场景写作工具
pub struct Veo3SceneWriter {
sdk: GeminiSDK,
agent_id: String,
/// 会话历史记录
conversation_history: Vec<String>,
}
/// VEO3 角色定义工具
pub struct Veo3ActorDefine {
sdk: GeminiSDK,
agent_id: String,
/// 会话历史记录
conversation_history: Vec<String>,
}
impl Veo3SceneWriter {
/// 创建新的 VEO3 场景写作实例
pub async fn new() -> Result<Self, Box<dyn std::error::Error>> {
let mut sdk = GeminiSDK::new()?;
// 创建 VEO3 场景写作 agent
let agent_id = "veo3-scene-writer";
let agent = Agent::new(
agent_id,
"VEO3 场景写作专家",
VEO3_SYSTEM_PROMPT
)
.with_temperature(0.7)
.with_max_tokens(8192);
sdk.add_agent(agent);
Ok(Self {
sdk,
agent_id: agent_id.to_string(),
conversation_history: Vec::new(),
})
}
/// 使用自定义配置创建实例
pub async fn with_config(config: gemini_sdk::GeminiConfig) -> Result<Self, Box<dyn std::error::Error>> {
let mut sdk = GeminiSDK::with_config(config)?;
// 创建 VEO3 场景写作 agent
let agent_id = "veo3-scene-writer";
let agent = Agent::new(
agent_id,
"VEO3 场景写作专家",
VEO3_SYSTEM_PROMPT
)
.with_temperature(0.7)
.with_max_tokens(8192);
sdk.add_agent(agent);
Ok(Self {
sdk,
agent_id: agent_id.to_string(),
conversation_history: Vec::new(),
})
}
/// 发送文本消息
pub async fn send_message(&mut self, message: &str) -> Result<String, Box<dyn std::error::Error>> {
// 构建包含历史记录的完整消息
let full_message = self.build_message_with_history(message);
let request = MessageRequest::with_agent(&full_message, &self.agent_id);
let response = self.sdk.send_message(request).await?;
// 保存到历史记录
self.add_to_history(message, &response.content);
Ok(response.content)
}
/// 发送带附件的消息
pub async fn send_message_with_attachments(
&mut self,
message: &str,
attachment_paths: Vec<&str>
) -> Result<String, Box<dyn std::error::Error>> {
// 构建包含历史记录的完整消息
let full_message = self.build_message_with_history(message);
let mut request = MessageRequest::with_agent(&full_message, &self.agent_id);
for path in attachment_paths {
request = request.attach(Attachment::new(path));
}
let response = self.sdk.send_message(request).await?;
// 保存到历史记录
self.add_to_history(message, &response.content);
Ok(response.content)
}
/// 发送带单个附件的消息
pub async fn send_message_with_attachment(
&mut self,
message: &str,
attachment_path: &str
) -> Result<String, Box<dyn std::error::Error>> {
self.send_message_with_attachments(message, vec![attachment_path]).await
}
/// 清除会话历史
pub fn clear_conversation(&mut self) {
self.conversation_history.clear();
}
/// 获取会话历史
pub fn get_conversation_history(&self) -> &Vec<String> {
&self.conversation_history
}
/// 构建包含历史记录的消息
fn build_message_with_history(&self, message: &str) -> String {
if self.conversation_history.is_empty() {
message.to_string()
} else {
// 获取最近的几轮对话作为上下文
let recent_history = self.conversation_history
.iter()
.rev()
.take(6) // 最近3轮对话用户+AI各3次
.rev()
.cloned()
.collect::<Vec<_>>()
.join("\n");
format!("对话历史:\n{}\n\n当前消息: {}", recent_history, message)
}
}
/// 添加到历史记录
fn add_to_history(&mut self, user_message: &str, ai_response: &str) {
self.conversation_history.push(format!("用户: {}", user_message));
self.conversation_history.push(format!("助手: {}", ai_response));
// 限制历史记录长度保留最近20条消息
if self.conversation_history.len() > 20 {
self.conversation_history.drain(0..self.conversation_history.len() - 20);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_veo3_scene_writer_creation() {
// 注意:这个测试需要有效的 API 配置才能通过
// 在实际环境中运行
if std::env::var("GEMINI_BEARER_TOKEN").is_ok() {
let writer = Veo3SceneWriter::new().await;
assert!(writer.is_ok());
}
}
#[tokio::test]
async fn test_veo3_actor_define_creation() {
// 注意:这个测试需要有效的 API 配置才能通过
// 在实际环境中运行
if std::env::var("GEMINI_BEARER_TOKEN").is_ok() {
let actor_define = Veo3ActorDefine::new().await;
assert!(actor_define.is_ok());
}
}
}
impl Veo3ActorDefine {
/// 创建新的 VEO3 角色定义实例
pub async fn new() -> Result<Self, Box<dyn std::error::Error>> {
let mut sdk = GeminiSDK::new()?;
// 创建 VEO3 角色定义 agent
let agent_id = "veo3-actor-define";
let agent = Agent::new(
agent_id,
"VEO3 角色定义专家",
VEO3_ACTOR_DEFINE_PROMPT
)
.with_temperature(0.7)
.with_max_tokens(8192);
sdk.add_agent(agent);
Ok(Self {
sdk,
agent_id: agent_id.to_string(),
conversation_history: Vec::new(),
})
}
/// 使用自定义配置创建实例
pub async fn with_config(config: gemini_sdk::GeminiConfig) -> Result<Self, Box<dyn std::error::Error>> {
let mut sdk = GeminiSDK::with_config(config)?;
// 创建 VEO3 角色定义 agent
let agent_id = "veo3-actor-define";
let agent = Agent::new(
agent_id,
"VEO3 角色定义专家",
VEO3_ACTOR_DEFINE_PROMPT
)
.with_temperature(0.7)
.with_max_tokens(8192);
sdk.add_agent(agent);
Ok(Self {
sdk,
agent_id: agent_id.to_string(),
conversation_history: Vec::new(),
})
}
/// 发送文本消息
pub async fn send_message(&mut self, message: &str) -> Result<String, Box<dyn std::error::Error>> {
// 构建包含历史记录的完整消息
let full_message = self.build_message_with_history(message);
let request = MessageRequest::with_agent(&full_message, &self.agent_id);
let response = self.sdk.send_message(request).await?;
// 保存到历史记录
self.add_to_history(message, &response.content);
Ok(response.content)
}
/// 发送带附件的消息
pub async fn send_message_with_attachments(
&mut self,
message: &str,
attachment_paths: Vec<&str>
) -> Result<String, Box<dyn std::error::Error>> {
// 构建包含历史记录的完整消息
let full_message = self.build_message_with_history(message);
let mut request = MessageRequest::with_agent(&full_message, &self.agent_id);
for path in attachment_paths {
request = request.attach(Attachment::new(path));
}
let response = self.sdk.send_message(request).await?;
// 保存到历史记录
self.add_to_history(message, &response.content);
Ok(response.content)
}
/// 发送带单个附件的消息
pub async fn send_message_with_attachment(
&mut self,
message: &str,
attachment_path: &str
) -> Result<String, Box<dyn std::error::Error>> {
self.send_message_with_attachments(message, vec![attachment_path]).await
}
/// 清除会话历史
pub fn clear_conversation(&mut self) {
self.conversation_history.clear();
}
/// 获取会话历史
pub fn get_conversation_history(&self) -> &Vec<String> {
&self.conversation_history
}
/// 构建包含历史记录的消息
fn build_message_with_history(&self, message: &str) -> String {
if self.conversation_history.is_empty() {
message.to_string()
} else {
// 获取最近的几轮对话作为上下文
let recent_history = self.conversation_history
.iter()
.rev()
.take(6) // 最近3轮对话用户+AI各3次
.rev()
.cloned()
.collect::<Vec<_>>()
.join("\n");
format!("对话历史:\n{}\n\n当前消息: {}", recent_history, message)
}
}
/// 添加到历史记录
fn add_to_history(&mut self, user_message: &str, ai_response: &str) {
self.conversation_history.push(format!("用户: {}", user_message));
self.conversation_history.push(format!("助手: {}", ai_response));
// 限制历史记录长度保留最近20条消息
if self.conversation_history.len() > 20 {
self.conversation_history.drain(0..self.conversation_history.len() - 20);
}
}
}