合并分支
* Merge branch 'main' into feature/modal-cluster-gemini * FIX 修复带图片模态推理接口 * PERF 优化Gemini Prompt词 * Merge remote-tracking branch 'origin/main' into feature/modal-cluster-gemini * ADD 增加网格图片创建接口 ADD gemini推理打点增加商品封面模态 PERF gemini打点结果相邻时间线同商品合并 * Merge branch 'main' into feature/modal-cluster-gemini * FIX 429重试机制 * FIX 修复Gemini推理函数入参 * Merge branch 'main' into feature/modal-cluster-gemini * PERF gemini修改参数类型 ADD 增加批量删除Gemini文件接口 * PERF gemini修改prompt * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * PERF gemini修改prompt * 录制HLS时 结果复制到S3挂载点增加一个fallback处理 * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * PERF gemini修改prompt * PERF gemini增加图像模态 * PERF gemini增加图像模态 * fix : TikHub的Logger换为Loguru * fix : 修复直播切分视频接口 * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * PERF tikhub接口增加随机延迟 * Merge remote-tracking branch 'origin/feature/modal-cluster' into feature/modal-cluster * 合并分支 * fix : hls录制缓存读写问题 * FIX 修改冲突 * ADD 增加获取直播间指定商品数量接口 * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * PERF 修改tikhub到aws * fix : 增加s3分片上传接口 * PERF Gemini推理直接从参数列表获取商品列表 ADD Tikhub增加获取置顶|讲解中商品接口 * PERF Gemini推理直接从参数获取商品列表 * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * fix 处理jpg图片的codec_name应为mjpg * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * PERF 优化返回值 * fix 获取媒体metadata的类型报错，modal client升级到1.0.2 * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * PERF 风控后爬虫直接退出避免cookie失效 PERF 分离tikhub api, 使用更低的保持时间以更换IP FIX 修复Dict键不存在报错的bug * fix 一些webhook相关的bug，添加了更详细的接口说明 * 更新直播录制为可跳转时间的hls流接口，用于替代掉腾讯VOD+火山云拉流转推，片段保证每片开头为关键帧，时间精度准确到毫秒级 * ADD 增加直播间商品增量缓存触发器 * Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini * PERF 调整gemini prompt * - KVCache类改为可拓展，基于环境变量设置KV space - test 环境配置与CF测试环境对齐 * FIX 修复Gemini推理视频切条的问题 PERF 优化视频切条函数性能 * ADD 基本完成gemini视频推理API * ADD 添加内联的tikhub * 对齐预发环境 --------- Merge request URL: https://g-ldyi2063.coding.net/p/dev/d/modalDeploy/git/merge/4794 Co-authored-by: 康宇佳，shuohigh@gmail.com
This commit is contained in:
Coding
BIN
simhei.ttf
Normal file
BIN
simhei.ttf
Normal file
Binary file not shown.
@@ -436,6 +436,7 @@ class FFMPEGStreamRecordRequest(BaseFFMPEGTaskRequest):
|
||||
|
||||
class GeminiRequest(BaseFFMPEGTaskRequest):
|
||||
media_hls_url: MediaSource = Field(default="", description="视频流录制HLS地址 hls://格式 需录制超过20分钟")
|
||||
product_cover_grid_uri_list: List[str] = Field(description="商品封面网格拼图URI列表")
|
||||
product_list: List[Union[str, dict]] = Field(description="商品名列表(时间倒序)"),
|
||||
start_time: str = Field(default="00:00:00.000", description="开始时间(hls)")
|
||||
end_time: str = Field(default="00:20:00.000", description="结束时间(hls)")
|
||||
@@ -454,7 +455,6 @@ class GeminiRequest(BaseFFMPEGTaskRequest):
|
||||
else:
|
||||
raise pydantic.ValidationError("media格式读取失败")
|
||||
|
||||
|
||||
class GeminiResultResponse(BaseFFMPEGTaskStatusResponse):
|
||||
result: str = Field(default="", description="推理出的json")
|
||||
|
||||
@@ -483,7 +483,14 @@ class MonitorLiveRoomProductRequest(BaseModel):
|
||||
|
||||
|
||||
class LiveRoomProductCachesResponse(BaseModel):
|
||||
status: int = Field(default=None,
|
||||
description="缓存状态/0-正常返回 1-直播已结束 2-IP风控 3-请求Tikhub API错误 4-内部错误")
|
||||
status: int = Field(default=None, description="缓存状态/0-正常返回 1-直播已结束 2-IP风控 3-请求Tikhub API错误 4-内部错误")
|
||||
message: str = Field(default="", description="错误信息")
|
||||
cache_json: str = Field(default="", description="缓存内容/Json文本")
|
||||
|
||||
class MakeGridGeminiRequest(BaseFFMPEGTaskRequest):
|
||||
pic_info_list: List[Dict[str, str]] = Field(default=[], description="包含图片信息的字典列表，每个字典包含 \"title\" 和 \"cover\" 键")
|
||||
image_size: int = Field(default=450, description="单个图片网格的尺寸/像素")
|
||||
text_height: int = Field(default=40, description="文本框的高度/像素")
|
||||
font_size: int = Field(default=18, description="文本尺寸/像素")
|
||||
padding: int = Field(default=5, description="文本距离文本框边缘距离/像素")
|
||||
separator: int = Field(default=12, description="分割线宽度/像素")
|
||||
@@ -275,7 +275,7 @@ async def s3_upload_base64(body: UploadBase64Request) -> UploadResultResponse:
|
||||
@router.post("/monitor_live_room_product_trigger",
|
||||
summary="触发监控直播间商品信息并缓存",
|
||||
description="触发监控直播间商品信息并缓存, 如果直播结束清除缓存, 触发间隔请控制在60s以上",
|
||||
dependencies=[Depends(verify_token)])
|
||||
dependencies=[Depends(verify_token)], deprecated=True)
|
||||
async def monitor_live_room_product(body: MonitorLiveRoomProductRequest) -> LiveRoomProductCachesResponse:
|
||||
fn = modal.Function.from_name(config.modal_app_name, "monitor_live_room_product_trigger",
|
||||
environment_name=config.modal_environment)
|
||||
|
||||
@@ -14,7 +14,7 @@ from starlette.responses import JSONResponse
|
||||
|
||||
from BowongModalFunctions.config import WorkerConfig
|
||||
from BowongModalFunctions.models.web_model import SentryTransactionInfo, GeminiResultResponse, GeminiRequest, \
|
||||
ModalTaskResponse
|
||||
ModalTaskResponse, MakeGridGeminiRequest
|
||||
from BowongModalFunctions.utils.ModalUtils import ModalUtils
|
||||
|
||||
config = WorkerConfig()
|
||||
@@ -25,7 +25,6 @@ router = APIRouter(prefix="/google", tags=["Google"])
|
||||
class GoogleAPIKeyHeaders(BaseModel):
|
||||
x_google_api_key: Optional[str] = Field(description="Google API Key")
|
||||
|
||||
|
||||
class BundleHeaders(BaseModel):
|
||||
x_google_api_key: Optional[str] = Field(description="Google API Key")
|
||||
x_trace_id: str = Field(description="Sentry Transaction ID", default=None)
|
||||
@@ -102,7 +101,6 @@ async def delete_file(filename: str, headers: Annotated[GoogleAPIKeyHeaders, Hea
|
||||
response.raise_for_status()
|
||||
return JSONResponse(content=response.json(), status_code=response.status_code)
|
||||
|
||||
|
||||
@router.delete('/delete_all', summary="删除所有已上传的文件/第一页")
|
||||
async def delete_all(headers: Annotated[GoogleAPIKeyHeaders, Header()]):
|
||||
google_api_key = headers.x_google_api_key or os.environ.get("GOOGLE_API_KEY")
|
||||
@@ -122,7 +120,6 @@ async def delete_all(headers: Annotated[GoogleAPIKeyHeaders, Header()]):
|
||||
return JSONResponse(content={"msg": f"删除文件{filename}失败"}, status_code=resp.status_code)
|
||||
return JSONResponse(content=response.json(), status_code=response.status_code)
|
||||
|
||||
|
||||
@router.get('/list', summary="列出已上传的文件")
|
||||
async def list_files(headers: Annotated[GoogleAPIKeyHeaders, Header()]):
|
||||
google_api_key = headers.x_google_api_key or os.environ.get("GOOGLE_API_KEY")
|
||||
@@ -135,6 +132,16 @@ async def list_files(headers: Annotated[GoogleAPIKeyHeaders, Header()]):
|
||||
return JSONResponse(content={}, status_code=response.status_code)
|
||||
return JSONResponse(content=response.json() if len(response.text) > 0 else "", status_code=response.status_code)
|
||||
|
||||
@router.post("/make_grid_gemini", summary="将输入图拼为网格上传到Gemini网盘")
|
||||
async def make_grid_gemini_upload(data: MakeGridGeminiRequest, headers: Annotated[BundleHeaders, Header()]):
|
||||
google_api_key = headers.x_google_api_key or os.environ.get("GOOGLE_API_KEY")
|
||||
if not google_api_key:
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing Google API Key")
|
||||
fn = modal.Function.from_name(config.modal_app_name,"make_image_grid_upload", environment_name=config.modal_environment)
|
||||
image_grid_uri = await fn.remote.aio(data.pic_info_list, data.image_size, data.text_height, data.font_size, data.padding, data.separator, google_api_key,
|
||||
SentryTransactionInfo(x_trace_id=sentry_sdk.get_traceparent(), x_baggage=sentry_sdk.get_baggage())
|
||||
if headers.x_trace_id is None else SentryTransactionInfo(x_trace_id=headers.x_trace_id, x_baggage=headers.x_baggage))
|
||||
return JSONResponse(content={"uri":image_grid_uri}, status_code=200)
|
||||
|
||||
@router.post('/inference_gemini', summary="使用Gemini推理hls视频流指定时间段的打点情况")
|
||||
async def inference_gemini(
|
||||
@@ -144,13 +151,10 @@ async def inference_gemini(
|
||||
google_api_key = headers.x_google_api_key or os.environ.get("GOOGLE_API_KEY")
|
||||
if not google_api_key:
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing Google API Key")
|
||||
fn = modal.Function.from_name(config.modal_app_name, "video_hls_slice_inference",
|
||||
environment_name=config.modal_environment)
|
||||
fn_call = fn.spawn(data.media_hls_url, google_api_key, data.product_list, data.start_time, data.end_time,
|
||||
SentryTransactionInfo(x_trace_id=sentry_sdk.get_traceparent(),
|
||||
x_baggage=sentry_sdk.get_baggage())
|
||||
if headers.x_trace_id is None else SentryTransactionInfo(x_trace_id=headers.x_trace_id,
|
||||
x_baggage=headers.x_baggage))
|
||||
fn = modal.Function.from_name(config.modal_app_name,"video_hls_slice_inference", environment_name=config.modal_environment)
|
||||
fn_call = fn.spawn(data.media_hls_url, google_api_key, data.product_cover_grid_uri_list, data.product_list, data.start_time, data.end_time,
|
||||
SentryTransactionInfo(x_trace_id=sentry_sdk.get_traceparent(), x_baggage=sentry_sdk.get_baggage())
|
||||
if headers.x_trace_id is None else SentryTransactionInfo(x_trace_id=headers.x_trace_id, x_baggage=headers.x_baggage))
|
||||
return ModalTaskResponse(success=True, taskId=fn_call.object_id)
|
||||
|
||||
|
||||
@@ -162,8 +166,7 @@ async def gemini_status(task_id: str, response: Response) -> GeminiResultRespons
|
||||
response.headers["x-baggage"] = task_info.transaction.x_baggage
|
||||
try:
|
||||
return GeminiResultResponse(taskId=task_id, status=task_info.status, code=cast(int, task_info.error_code.value),
|
||||
error=task_info.error_reason,
|
||||
result=json.dumps(task_info.results, ensure_ascii=False)
|
||||
error=task_info.error_reason, result=json.dumps(task_info.results, ensure_ascii=False)
|
||||
if task_info.results is not None else "")
|
||||
except Exception as e:
|
||||
logger.exception(f"获取Gemini状态发生错误 {e}")
|
||||
|
||||
@@ -14,3 +14,111 @@ class TimeDelta(timedelta):
|
||||
|
||||
def toFormatStr(self) -> str:
|
||||
return (datetime(year=2000, month=1, day=1) + self).strftime("%H:%M:%S.%f")[:-3]
|
||||
|
||||
|
||||
def parse_time(time_str):
|
||||
"""解析时间字符串为datetime对象，支持两种格式：HH:MM:SS.fff 和 MM:SS.fff"""
|
||||
# 尝试按两种格式解析
|
||||
try:
|
||||
# 先尝试完整格式 HH:MM:SS.fff
|
||||
return datetime.strptime(time_str, '%H:%M:%S.%f')
|
||||
except ValueError:
|
||||
try:
|
||||
# 如果失败，尝试 MM:SS.fff 格式
|
||||
dt = datetime.strptime(time_str, '%M:%S.%f')
|
||||
# 将小时设为0，只保留分钟和秒
|
||||
return datetime.combine(dt.date(), dt.time().replace(hour=0))
|
||||
except ValueError:
|
||||
try:
|
||||
# 如果失败，尝试 MM:SS:fff 格式
|
||||
dt = datetime.strptime(time_str, '%M:%S:%f')
|
||||
# 将小时设为0，只保留分钟和秒
|
||||
return datetime.combine(dt.date(), dt.time().replace(hour=0))
|
||||
except ValueError:
|
||||
# 如果失败，尝试 HH:MM:SS:fff 格式
|
||||
return datetime.strptime(time_str, '%H:%M:%S:%f')
|
||||
|
||||
|
||||
def format_time(dt):
|
||||
"""将datetime对象格式化为时间字符串"""
|
||||
return dt.strftime('%H:%M:%S.%f')[:-3] # 保留3位小数
|
||||
|
||||
|
||||
def parse_timeline_item(item):
|
||||
"""解析时间线项目，提取开始和结束时间"""
|
||||
time_range, _ = item.split(' (', 1)
|
||||
start_time_str, end_time_str = time_range.split(' - ')
|
||||
return parse_time(start_time_str), parse_time(end_time_str)
|
||||
|
||||
|
||||
def format_timeline_item(start_time, end_time, activity):
|
||||
"""格式化时间线项目"""
|
||||
return f"{format_time(start_time)} - {format_time(end_time)} ({activity})"
|
||||
|
||||
|
||||
def merge_timeline_items(items):
|
||||
"""合并相邻的时间线项目，保留不同的活动描述"""
|
||||
if not items:
|
||||
return []
|
||||
|
||||
# 解析所有项目
|
||||
parsed_items = []
|
||||
for item in items:
|
||||
start_time, end_time = parse_timeline_item(item)
|
||||
activity = item.split(' (', 1)[1].rstrip(')')
|
||||
parsed_items.append((start_time, end_time, activity))
|
||||
|
||||
# 按开始时间排序
|
||||
parsed_items.sort(key=lambda x: x[0])
|
||||
|
||||
# 合并相邻的时间段
|
||||
merged_items = [parsed_items[0]]
|
||||
for current in parsed_items[1:]:
|
||||
last = merged_items[-1]
|
||||
|
||||
# 如果当前项目的开始时间与上一个项目的结束时间相邻或重叠
|
||||
if current[0] <= last[1]:
|
||||
# 更新结束时间为两个结束时间的最大值
|
||||
new_end_time = max(last[1], current[1])
|
||||
|
||||
# 检查活动描述是否相同
|
||||
if last[2] == current[2]:
|
||||
# 活动描述相同，直接合并
|
||||
merged_items[-1] = (last[0], new_end_time, last[2])
|
||||
else:
|
||||
# 活动描述不同，需要保留两个时间段
|
||||
# 先分割上一个项目
|
||||
if last[1] < current[1]:
|
||||
# 当前项目的结束时间更晚，需要分割
|
||||
merged_items[-1] = (last[0], current[0], last[2])
|
||||
merged_items.append((current[0], new_end_time, current[2]))
|
||||
else:
|
||||
# 上一个项目的结束时间更晚，需要分割
|
||||
merged_items[-1] = (last[0], current[0], last[2])
|
||||
merged_items.append((current[0], current[1], current[2]))
|
||||
merged_items.append((current[1], last[1], last[2]))
|
||||
else:
|
||||
# 不相邻，添加新项目
|
||||
merged_items.append(current)
|
||||
|
||||
# 格式化回原始字符串格式
|
||||
return [format_timeline_item(start, end, activity) for start, end, activity in merged_items]
|
||||
|
||||
|
||||
def merge_product_data(data):
|
||||
"""合并相同产品的数据"""
|
||||
product_dict = {}
|
||||
|
||||
# 按产品名称分组
|
||||
for item in data:
|
||||
product = item["product"]
|
||||
if product not in product_dict:
|
||||
product_dict[product] = {"product": product, "timeline": []}
|
||||
product_dict[product]["timeline"].extend(item["timeline"])
|
||||
|
||||
# 合并每个产品的时间线
|
||||
for product in product_dict:
|
||||
product_dict[product]["timeline"] = merge_timeline_items(product_dict[product]["timeline"])
|
||||
|
||||
# 返回合并后的列表
|
||||
return list(product_dict.values())
|
||||
@@ -1,15 +1,25 @@
|
||||
import base64
|
||||
import math
|
||||
import random
|
||||
import subprocess
|
||||
import uuid
|
||||
from io import BytesIO
|
||||
|
||||
import modal
|
||||
from asyncio import sleep
|
||||
|
||||
from PIL import ImageDraw, Image, ImageFont
|
||||
from dotenv import dotenv_values
|
||||
|
||||
from BowongModalFunctions.utils.SentryUtils import SentryUtils
|
||||
|
||||
downloader_image = (
|
||||
modal.Image
|
||||
.debian_slim(python_version="3.11")
|
||||
.apt_install('ffmpeg')
|
||||
.pip_install_from_pyproject("../pyproject.toml")
|
||||
.add_local_file("../simhei.ttf", "/usr/share/fonts/simhei.ttf", copy=True)
|
||||
.run_commands("chmod -R 777 /usr/share/fonts")
|
||||
.env(dotenv_values("../.runtime.env"))
|
||||
.add_local_python_source('cluster')
|
||||
.add_local_python_source('BowongModalFunctions')
|
||||
@@ -28,7 +38,7 @@ with downloader_image.imports():
|
||||
import sentry_sdk
|
||||
from sentry_sdk.integrations.loguru import LoguruIntegration
|
||||
from tqdm import tqdm
|
||||
from typing import Tuple, List
|
||||
from typing import Tuple, List, Dict
|
||||
from loguru import logger
|
||||
from datetime import datetime, UTC, timedelta, timezone
|
||||
from modal import current_function_call_id
|
||||
@@ -45,9 +55,9 @@ with downloader_image.imports():
|
||||
from BowongModalFunctions.utils.KVCache import MediaSourceKVCache, LiveProductKVCache
|
||||
from BowongModalFunctions.models.media_model import MediaSource, MediaCacheStatus, MediaProtocol
|
||||
from BowongModalFunctions.models.web_model import SentryTransactionInfo, MonitorLiveRoomProductRequest, LiveProduct, \
|
||||
LiveProductCaches
|
||||
LiveProductCaches, WebhookNotify
|
||||
from BowongModalFunctions.models.ffmpeg_worker_model import FFMpegSliceSegment
|
||||
from BowongModalFunctions.utils.TimeUtils import TimeDelta
|
||||
from BowongModalFunctions.utils.TimeUtils import TimeDelta, merge_product_data
|
||||
|
||||
config = WorkerConfig()
|
||||
|
||||
@@ -347,10 +357,12 @@ with downloader_image.imports():
|
||||
@modal.concurrent(max_inputs=1)
|
||||
async def video_hls_slice_inference(media: MediaSource,
|
||||
google_api_key: str,
|
||||
product_grid_list: List[str],
|
||||
product_list: List[any],
|
||||
start_time: str,
|
||||
end_time: str,
|
||||
sentry_trace: SentryTransactionInfo,
|
||||
webhook: WebhookNotify=None,
|
||||
retry_time: int=3
|
||||
):
|
||||
|
||||
@@ -451,40 +463,38 @@ with downloader_image.imports():
|
||||
|
||||
return upload_response.json(), upload_response.status_code
|
||||
|
||||
@SentryUtils.webhook_handler(webhook, current_function_call_id())
|
||||
@SentryUtils.sentry_tracker(sentry_trace.x_trace_id, sentry_trace.x_baggage, op="inference_gemini", name="Gemini推理", fn_id=current_function_call_id())
|
||||
async def _handler(media: MediaSource,
|
||||
google_api_key: str,
|
||||
product_grid_list: List[str],
|
||||
product_list: List[any],
|
||||
start_time: str,
|
||||
end_time: str,
|
||||
sentry_trace: SentryTransactionInfo,
|
||||
retry_time: int=3):
|
||||
|
||||
video_gemini_uri = None
|
||||
|
||||
try:
|
||||
# 1、首先获取全量商品列表
|
||||
# try:
|
||||
logger.info("1、获取直播间商品列表")
|
||||
# product_list = modal_kv_product_cache.get_cache(room_id)
|
||||
# if not product_list:
|
||||
# raise Exception(f"room_id {room_id} 商品列表缓存为空, 请先发起请求缓存商品列表")
|
||||
# product_list = product_list.product_list
|
||||
# logger.success(f"获取商品列表成功, 长度{len(product_list)}")
|
||||
# except Exception as e:
|
||||
# logger.exception(f"获取商品列表失败：{e}")
|
||||
# raise
|
||||
if len(product_list) == 0:
|
||||
logger.error("商品列表为空, 退出推理")
|
||||
raise Exception("商品列表为空, 退出推理")
|
||||
product_title_list = []
|
||||
product_cover_list = []
|
||||
if isinstance(product_list[0], dict):
|
||||
for product in product_list:
|
||||
product_title_list.append(product["title"])
|
||||
product_cover_list.append(product["cover"])
|
||||
else:
|
||||
product_title_list = product_list
|
||||
for _ in product_list:
|
||||
product_cover_list.append("")
|
||||
product_json = []
|
||||
for title, cover in zip(product_title_list, product_cover_list):
|
||||
product_json.append({
|
||||
"product_name": title,
|
||||
"product_cover": cover,
|
||||
})
|
||||
logger.info(json.dumps(product_json, ensure_ascii=False, indent=2))
|
||||
logger.info("product_title_list: \n" + "\n".join(product_title_list))
|
||||
image_parts=[{
|
||||
"file_data": {
|
||||
"mime_type": "image/jpeg",
|
||||
"file_uri": f"{i}"
|
||||
}
|
||||
} for i in product_grid_list]
|
||||
# 2、切20分钟的条
|
||||
logger.info("2、开始截取指定视频")
|
||||
slice_fn = modal.Function.from_name(config.modal_app_name,"ffmpeg_slice_media", environment_name=config.modal_environment)
|
||||
@@ -530,23 +540,8 @@ with downloader_image.imports():
|
||||
def inference_api():
|
||||
try:
|
||||
logger.info("请求推理接口")
|
||||
resp = requests.post(
|
||||
"https://gateway.ai.cloudflare.com/v1/67720b647ff2b55cf37ba3ef9e677083/bowong-dev/google-ai-studio/v1beta/models/gemini-2.5-flash-preview-05-20:generateContent",
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"X-Goog-Api-Key": f"{google_api_key}"
|
||||
},
|
||||
json={
|
||||
"safetySettings": [
|
||||
{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_CIVIC_INTEGRITY", "threshold": "BLOCK_NONE"}
|
||||
],
|
||||
"contents": [
|
||||
{
|
||||
"parts": [
|
||||
product_list_str = "，".join(product_title_list)
|
||||
image_parts.extend([
|
||||
{
|
||||
"file_data": {
|
||||
"mime_type": "video/mp4",
|
||||
@@ -556,25 +551,85 @@ with downloader_image.imports():
|
||||
{
|
||||
"text": """<prompt>
|
||||
<instruction>
|
||||
我上传的是一个主播直播卖货的片段，帮我分析这个视频里的画面、语音、字幕，然后归纳其中的核心讲解的商品，注意不要包含任何不安全或色情的内容。
|
||||
我上传了一个主播直播卖货的视频片段和一些商品图片。每张图片中包含多个服装商品，每个商品图上方都有完整的商品名标注，且均可以在商品列表找到对应的商品名。
|
||||
|
||||
**❗重要：图片中没有的商品绝对不能输出❗**
|
||||
|
||||
**商品识别的铁律**：
|
||||
- 如果商品只在商品列表中存在，但图片中看不到，绝对禁止输出
|
||||
- 只有图片中实际能看到的商品，才允许在结果中出现
|
||||
- 宁可漏掉商品，也不能虚构图片中不存在的商品
|
||||
|
||||
**商品名称匹配规则**：
|
||||
1. 首先仔细观察图片，确认图片中实际存在哪些商品
|
||||
2. 然后观察这些商品的外观特征（颜色、款式）
|
||||
3. 在视频中识别主播介绍的商品时，只能从图片中实际存在的商品中选择
|
||||
4. 商品名称必须使用商品列表中的准确名称，但前提是该商品在图片中确实存在
|
||||
|
||||
**商品列表（仅供命名参考）：**
|
||||
{0}
|
||||
|
||||
**重要提醒**：图片中可能包含同款式但不同颜色的商品，在匹配视频内容时，必须准确识别视频中商品的颜色和款式，然后在商品列表中找到对应的商品名称。
|
||||
|
||||
按照以下json格式返回：
|
||||
[{"timeline":["核心商品所属时间段落"],"product":"核心商品名"}]
|
||||
[{{"timeline":["核心商品所属时间段落"],"product":"核心商品名"}}]
|
||||
</instruction>
|
||||
|
||||
<datas>
|
||||
<!-- 核心商品列表 -->
|
||||
<data>{"product_name":%s}</data>
|
||||
</datas>
|
||||
|
||||
<rules>
|
||||
<!-- 核心商品判断标准 -->
|
||||
<rule>参考（核心商品列表）里的商品名称信息（必须从product_name里的列表中取值），找出他们中谁在这段视频中有过介绍或者穿着。</rule>
|
||||
<!-- 🚨 最高优先级铁律 🚨 -->
|
||||
<rule>**绝对禁止虚构商品**：
|
||||
- 这是最重要的规则，违反此规则的分析结果完全无效
|
||||
- 图片中没有的商品，无论商品列表中是否存在，都绝对不能出现在JSON结果中
|
||||
- 每输出一个商品前，必须在图片中找到对应的实物证据
|
||||
- 如果不确定图片中是否有某个商品，宁可不输出也不要冒险输出</rule>
|
||||
|
||||
<!-- 商品名称识别规则 -->
|
||||
<rule>**商品名称匹配优先级**：
|
||||
1. 必须使用商品列表中提供的准确商品名称，不得修改、简化或重新命名
|
||||
2. 通过观察图片中商品的外观特征（颜色、款式、设计细节）来识别对应的商品
|
||||
3. 将图片中的商品外观与视频中主播展示的商品进行匹配
|
||||
4. 最终输出的商品名称必须严格来自于提供的商品列表</rule>
|
||||
<rule>**商品识别流程**：
|
||||
1. **第一步：扫描图片** - 仔细观察图片中实际存在的所有商品及其外观特征（颜色、款式、图案、剪裁等）
|
||||
2. **第二步：分析视频** - 在视频中找出主播穿着或介绍的商品
|
||||
3. **第三步：匹配验证** - 将视频中的商品与图片中的商品进行匹配
|
||||
4. **第四步：名称确认** - 只有在图片中找到对应商品后，才从商品列表中选择最匹配的商品名称
|
||||
5. **第五步：最终检查** - 确保输出的每个商品都能在图片中找到实际对应的外观</rule>
|
||||
<rule>**严禁使用列表外商品名**：输出的商品名称必须100%来自提供的商品列表，不得使用图片中的文字标注或其他任何名称。</rule>
|
||||
<rule>**同款不同色识别**：特别注意商品列表中可能存在同款式但不同配色的商品。"同款式"的定义包括：
|
||||
- 纯色款的不同颜色（如同一T恤的白色版、黑色版、蓝色版）
|
||||
- 图案花纹相同但配色不同（如同样鱼图案，但一个是紫色压黄色，另一个是红色压蓝色）
|
||||
- 剪裁版型相同但印花配色不同（如同样条纹图案，但条纹颜色组合不同）
|
||||
在匹配时必须：
|
||||
- 首先识别图案/花纹的形状和设计是否相同
|
||||
- 然后对比具体的颜色搭配和配色方案
|
||||
- 如果视频中涉及多个同款不同配色的商品，都要识别出来，不要删除任何一个
|
||||
- 相同款式不同配色也算同一个商品</rule>
|
||||
<rule>**同款商品完整保留**：如果视频中主播穿着或介绍了商品列表中的多个同款不同色商品，必须在结果中保留所有相关商品，不能因为是同款就只选择其中一个。每个配色版本都有其独立的商业价值，都需要完整记录。</rule>
|
||||
<rule>**图案配色匹配精确度**：在判断商品匹配时要分层次考虑：
|
||||
- **第一层：图案形状匹配** - 花纹、印花、图案的形状和设计必须相同（如都是鱼图案、都是条纹、都是几何图形等）
|
||||
- **第二层：配色方案匹配** - 在图案形状相同的基础上，对比具体的颜色搭配：
|
||||
* 纯色款：主色调的准确匹配（如白色、黑色、蓝色、粉色等）
|
||||
* 印花款：印花颜色与底色的组合匹配（如紫色鱼+黄色底 vs 红色鱼+蓝色底）
|
||||
* 多色款：各颜色的位置和比例关系匹配
|
||||
- **容错处理** - 光线和拍摄角度可能造成的颜色偏差，但主要配色方案应该明显一致</rule>
|
||||
<rule>**图案款式智能识别**：对于有图案印花的商品，重点关注：
|
||||
- 如果图案的形状、设计、构图完全相同，但颜色搭配不同，这属于同一款式的不同配色
|
||||
- 例如：同样的鱼图案，一个是"紫色鱼+黄色底"，另一个是"红色鱼+蓝色底"，这是同款不同配色
|
||||
- 例如：同样的条纹图案，一个是"黑白条纹"，另一个是"蓝白条纹"，这是同款不同配色
|
||||
- 在识别时，先识别图案本身，再匹配具体的配色方案</rule>
|
||||
<rule>**商品范围严格限制**：
|
||||
- **双重验证机制**：商品必须同时满足两个条件才能输出：
|
||||
1. 商品名称必须存在于提供的商品列表中
|
||||
2. 商品外观必须能在图片中找到对应的实物
|
||||
- **图片优先原则**：如果商品列表中有某个商品，但图片中没有这个商品的外观，绝对不能输出这个商品
|
||||
- **严禁虚构商品**：不得基于商品列表推测或猜测图片中可能存在但实际看不到的商品</rule>
|
||||
<rule>一个商品可以存在多个timeline，请确保这个商品的所有出现过的时间都被识别出来，可以合并的碎片时间段尽量合并。</rule>
|
||||
<rule>如果一个商品的演示画面中，有多个商品，需要将这些商品都识别出来，分开到不同的商品中。这部分要精细一些</rule>
|
||||
<rule>如果一个片段，有主播不在画面内的内容，需要从这个商品片段中去除。</rule>
|
||||
|
||||
<!-- 穿着与介绍不一致的处理规则 -->
|
||||
<rule>特别注意区分主播穿着的商品和正在介绍的商品，有时主播会穿着A商品但介绍B商品，注意观察主播在与哪件衣服互动，也作为区分产品介绍的判断，注意主播经常穿搭一套服装，然后分别介绍不同的商品</rule>
|
||||
<rule>特别注意区分主播穿着的商品和正在介绍的商品，有时主播会穿着A商品但介绍B商品，注意观察主播在与哪个商品互动，也作为区分商品介绍的判断，注意主播经常穿搭一套服装，然后分别介绍不同的商品</rule>
|
||||
<rule>对于每个商品的时间段，需要明确该时间段内主播是否在介绍该商品本身，还是仅仅穿着该商品但介绍其他内容。</rule>
|
||||
|
||||
<!-- 切换商品的判断条件 -->
|
||||
@@ -582,30 +637,56 @@ with downloader_image.imports():
|
||||
<rule>结合画面、穿着，语音综合判断商品是否转换，不要只从某个维度进行判断。</rule>
|
||||
|
||||
<!-- timeline时间段规则 -->
|
||||
<rule>确保核心商品输出的timeline时间段落必须是该格式(小时:分钟:秒.毫秒)hh:mm:ss.mmm - hh:mm:ss.mmm。</rule>
|
||||
<rule>确保核心商品输出的timeline时间段落必须是该格式(小时:分钟:秒.毫秒)hh:mm:ss.mmm - hh:mm:ss.mmm。</ruhh
|
||||
<rule>每个时间段后面必须添加内容类型标识，格式为 (类型)，具体类型包括：</rule>
|
||||
<rule>• (穿着本品+介绍本品) - 主播穿着该商品且正在介绍该商品本身</rule>
|
||||
<rule>• (穿着本品+他品) - 主播穿着该商品但正在介绍其他商品</rule>
|
||||
<rule>• (穿着本品+无关) - 主播穿着该商品但在做无关商品的事情（如聊天、互动等）</rule>
|
||||
<rule>判断时要仔细分析主播的语音内容，确定其介绍的具体是哪个商品，避免混淆。</rule>
|
||||
|
||||
<!-- 🔍 输出前的最终检查 -->
|
||||
<rule>**输出前的强制检查（最重要）**：
|
||||
- 在生成JSON结果前，对每个商品进行最后检查
|
||||
- 自问："我能在上传的图片中明确看到这个商品的实物吗？"
|
||||
- 如果答案不是100%肯定的"是"，立即从结果中删除该商品
|
||||
- 宁可输出空数组[]，也绝不输出图片中不存在的商品
|
||||
- 这是检验分析质量的最终标准</rule>
|
||||
</rules>
|
||||
|
||||
<examples>
|
||||
<!-- 正确的输出格式 -->
|
||||
<example>
|
||||
[{"timeline":["hh:mm:ss.mmm - hh:mm:ss.mmm (穿着本品+介绍本品)","hh:mm:ss.mmm - hh:mm:ss.mmm (穿着本品+他品)","hh:mm:ss.mmm - hh:mm:ss.mmm (穿着本品+无关)"],"product":"蓝色连衣裙"}]
|
||||
[{{"timeline":["hh:mm:ss.mmm - hh:mm:ss.mmm (穿着本品+介绍本品)","hh:mm:ss.mmm - hh:mm:ss.mmm (穿着本品+他品)","hh:mm:ss.mmm - hh:mm:ss.mmm (穿着本品+无关)"],"product":"美洋MEIYANG【商场同款】水面膜T恤 莱赛尔修身圆领短袖-周一"}}]
|
||||
</example>
|
||||
<reason>正确区分了主播穿着蓝色连衣裙时的不同状态：第一段在介绍连衣裙本身，第二段穿着连衣裙但在介绍其他商品，第三段穿着连衣裙但在做无关事情。</reason>
|
||||
<reason>正确区分了主播穿着该T恤时的不同状态：第一段在介绍T恤本身，第二段穿着T恤但在介绍其他商品，第三段穿着T恤但在做无关事情。商品名称严格来自提供的商品列表："美洋MEIYANG【商场同款】水面膜T恤 莱赛尔修身圆领短袖-周一"。**关键：该商品必须在图片中实际可见**，通过对比图片中该商品的外观特征（颜色、款式）与视频中主播展示的商品来确定匹配关系。如果图片中没有这件T恤，绝不能输出。</reason>
|
||||
</examples>
|
||||
</prompt>""" % ([i for i in product_title_list],)
|
||||
</prompt>""".format(product_list_str)
|
||||
}
|
||||
]
|
||||
])
|
||||
json_data = {
|
||||
"safetySettings": [
|
||||
{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_NONE"},
|
||||
{"category": "HARM_CATEGORY_CIVIC_INTEGRITY", "threshold": "BLOCK_NONE"}
|
||||
],
|
||||
"contents": [
|
||||
{
|
||||
"parts": image_parts
|
||||
}
|
||||
],
|
||||
"generation_config": {
|
||||
"temperature": 0.1,
|
||||
}
|
||||
}, timeout=900
|
||||
}
|
||||
resp = requests.post(
|
||||
"https://gateway.ai.cloudflare.com/v1/67720b647ff2b55cf37ba3ef9e677083/bowong-dev/google-ai-studio/v1beta/models/gemini-2.5-flash-preview-05-20:generateContent",
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"X-Goog-Api-Key": f"{google_api_key}"
|
||||
},
|
||||
json=json_data, timeout=900
|
||||
)
|
||||
except Exception as e:
|
||||
logger.exception(f"Gemini推理请求失败: {e}")
|
||||
@@ -632,6 +713,8 @@ with downloader_image.imports():
|
||||
parts:str = target_json["candidates"][0]["content"]["parts"][0]["text"]
|
||||
parts = parts.replace("```","").replace("json\n","").replace("\n","").replace("\\","")
|
||||
parts = json.loads(parts)
|
||||
# 合并产品和时间线
|
||||
parts = merge_product_data(parts)
|
||||
return parts, sentry_trace
|
||||
else:
|
||||
raise Exception(f"Gemini推理失败, 失败原因: {reason}")
|
||||
@@ -650,6 +733,290 @@ with downloader_image.imports():
|
||||
logger.success("Gemini临时文件清除成功")
|
||||
else:
|
||||
logger.warning("Gemini临时文件清除失败, 请自行清除")
|
||||
return await _handler(media, google_api_key, product_grid_list, product_list, start_time, end_time, sentry_trace, retry_time)
|
||||
|
||||
@app.function(max_containers=config.video_downloader_concurrency, timeout=60)
|
||||
@modal.concurrent(max_inputs=200)
|
||||
async def make_image_grid_upload(pic_info_list:List[Dict[str,str]],
|
||||
image_size:int,
|
||||
text_height:int,
|
||||
font_size:int,
|
||||
padding:int,
|
||||
separator:int,
|
||||
google_api_key: str,
|
||||
sentry_trace:SentryTransactionInfo) -> str:
|
||||
def create_image_grid(image_info_list: List[Dict[str, str]],
|
||||
output_path: str,
|
||||
image_size: int = 450,
|
||||
text_height: int = 40,
|
||||
font_size: int = 18,
|
||||
padding: int = 5,
|
||||
separator: int = 5) -> str | None:
|
||||
"""
|
||||
创建一个包含图片和文字说明的马赛克拼图，最大36张图
|
||||
|
||||
:param image_info_list: 包含图片信息的字典列表，每个字典包含 "title" 和 "cover" 键
|
||||
:param output_path: 输出图片的保存路径
|
||||
:param image_size: 单个图片网格的尺寸/像素
|
||||
:param text_height: 文本框的高度/像素
|
||||
:param font_size: 文本尺寸/像素
|
||||
:param padding: 文本距离文本框边缘距离/像素
|
||||
:param separator: 分割线宽度/像素
|
||||
:return: 图片路径
|
||||
"""
|
||||
try:
|
||||
cell_height = image_size + text_height
|
||||
cell_width = image_size
|
||||
# 提取图片路径和文字说明
|
||||
image_paths = []
|
||||
captions = []
|
||||
for info in image_info_list:
|
||||
captions.append(info["title"])
|
||||
if "resize:200" in info["cover"]:
|
||||
info["cover"] = info["cover"].replace("resize:200", f"resize:{image_size}")
|
||||
image_paths.append(info["cover"])
|
||||
|
||||
# 检查输入
|
||||
if len(image_paths) != len(captions):
|
||||
raise ValueError("图片数量与文字说明数量不匹配")
|
||||
|
||||
# 处理图片（包括从网络获取）
|
||||
loaded_images = []
|
||||
for path in image_paths:
|
||||
if path.startswith(('http://', 'https://')):
|
||||
try:
|
||||
response = requests.get(path)
|
||||
response.raise_for_status()
|
||||
img = Image.open(BytesIO(response.content))
|
||||
loaded_images.append(img)
|
||||
except requests.RequestException as e:
|
||||
raise FileNotFoundError(f"无法从网络获取图片: {path}, 错误: {e}")
|
||||
elif os.path.exists(path):
|
||||
img = Image.open(path)
|
||||
loaded_images.append(img)
|
||||
else:
|
||||
raise FileNotFoundError(f"找不到图片文件: {path}")
|
||||
|
||||
# 计算网格大小
|
||||
n = len(image_paths)
|
||||
max_cols = min(6, n) # 最大列数为6或图片数量
|
||||
min_rows = math.ceil(n / max_cols) # 最小行数
|
||||
|
||||
# 找到最接近正方形的网格布局
|
||||
best_ratio = float('inf')
|
||||
best_rows, best_cols = min_rows, max_cols
|
||||
|
||||
for rows in range(min_rows, 7): # 最多6行
|
||||
cols = math.ceil(n / rows)
|
||||
if cols > 6: # 列数不能超过6
|
||||
continue
|
||||
|
||||
ratio = abs(rows / cols - 1) # 越接近1越好
|
||||
if ratio < best_ratio:
|
||||
best_ratio = ratio
|
||||
best_rows, best_cols = rows, cols
|
||||
|
||||
# 检查是否超出最大网格限制
|
||||
if best_rows > 6 or best_cols > 6:
|
||||
raise ValueError(f"图片数量({n})过多，无法在6x6网格内合理展示")
|
||||
|
||||
# 创建画布
|
||||
canvas_width = best_cols * (cell_width + separator) - separator
|
||||
canvas_height = best_rows * (cell_height + separator) - separator
|
||||
canvas = Image.new('RGB', (canvas_width, canvas_height), color='white')
|
||||
|
||||
# 优化分割线绘制 - 先绘制所有分割线
|
||||
draw = ImageDraw.Draw(canvas)
|
||||
|
||||
# 绘制垂直分割线
|
||||
for col in range(best_cols - 1):
|
||||
x = (col + 1) * (cell_width + separator) - separator // 2
|
||||
draw.line([(x, 0), (x, canvas_height)], fill=(0, 0, 0), width=separator)
|
||||
|
||||
# 绘制水平分割线
|
||||
for row in range(best_rows - 1):
|
||||
y = (row + 1) * (cell_height + separator) - separator // 2
|
||||
draw.line([(0, y), (canvas_width, y)], fill=(0, 0, 0), width=separator)
|
||||
|
||||
# 尝试加载支持中文的字体
|
||||
font = None
|
||||
# 尝试常用的中文字体
|
||||
chinese_fonts = [
|
||||
"simhei.ttf", # 黑体
|
||||
"simsun.ttc", # 宋体
|
||||
"microsoftyahei.ttf", # 微软雅黑
|
||||
"arial.ttf" # 最后尝试Arial
|
||||
]
|
||||
|
||||
for font_name in chinese_fonts:
|
||||
try:
|
||||
font = ImageFont.truetype(font_name, font_size)
|
||||
break
|
||||
except IOError:
|
||||
continue
|
||||
|
||||
# 如果所有字体都失败，使用默认字体
|
||||
if font is None:
|
||||
font = ImageFont.load_default()
|
||||
print("警告: 无法加载任何指定字体，使用默认字体")
|
||||
|
||||
# 绘制图片和文字
|
||||
for i, (img, caption) in enumerate(zip(loaded_images, captions)):
|
||||
row = i // best_cols
|
||||
col = i % best_cols
|
||||
|
||||
# 计算位置
|
||||
x = col * (cell_width + separator)
|
||||
y = row * (cell_height + separator)
|
||||
|
||||
# 保持原始长宽比
|
||||
img_width, img_height = img.size
|
||||
|
||||
# 计算等比例缩放后的尺寸
|
||||
ratio = min(image_size / img_width, image_size / img_height)
|
||||
new_width = int(img_width * ratio)
|
||||
new_height = int(img_height * ratio)
|
||||
|
||||
# 调整图片大小
|
||||
img = img.resize((new_width, new_height), Image.LANCZOS)
|
||||
|
||||
# 计算图片在单元格内的居中位置（只考虑图片区域，不包括文字区域）
|
||||
img_x = x + (cell_width - new_width) // 2
|
||||
|
||||
# 计算图片区域的起始位置（文字区域下方）
|
||||
image_area_start_y = y + min(text_height, text_height)
|
||||
|
||||
# 计算图片区域的高度（单元格高度减去文字区域高度）
|
||||
image_area_height = cell_height - (image_area_start_y - y)
|
||||
|
||||
# 计算图片在图片区域内的垂直居中位置
|
||||
img_y = image_area_start_y + (image_area_height - new_height) // 2
|
||||
|
||||
# 确保图片不会超出网格底部
|
||||
if img_y + new_height > y + cell_height:
|
||||
img_y = y + cell_height - new_height
|
||||
|
||||
# 将图片粘贴到画布上
|
||||
canvas.paste(img, (img_x, img_y))
|
||||
|
||||
# 添加文字区域 (透明背景)
|
||||
draw = ImageDraw.Draw(canvas)
|
||||
|
||||
# 改进的文字自动换行处理（支持中文）
|
||||
max_width = cell_width - 2 * padding
|
||||
lines = []
|
||||
current_line = ""
|
||||
|
||||
for char in caption:
|
||||
test_line = current_line + char
|
||||
left, top, right, bottom = font.getbbox(test_line)
|
||||
test_width = right - left
|
||||
|
||||
if test_width <= max_width:
|
||||
current_line = test_line
|
||||
else:
|
||||
lines.append(current_line)
|
||||
current_line = char
|
||||
|
||||
if current_line:
|
||||
lines.append(current_line)
|
||||
|
||||
# 计算文字总行高
|
||||
line_height = font_size # 使用固定行高
|
||||
total_text_height = len(lines) * line_height
|
||||
|
||||
# 绘制多行文字 - 不绘制背景，直接绘制文字
|
||||
for j, line in enumerate(lines):
|
||||
left, top, right, bottom = font.getbbox(line)
|
||||
text_width = right - left
|
||||
|
||||
# 水平居中
|
||||
text_x = x + padding + (cell_width - 2 * padding - text_width) // 2
|
||||
|
||||
# 垂直位置 - 从单元格顶部开始，考虑padding
|
||||
text_y_offset = y + padding + j * line_height
|
||||
|
||||
# 绘制文字边框
|
||||
offsets = [(-1, -1), (0, -1), (1, -1), (-1, 0), (1, 0), (-1, 1), (0, 1), (1, 1)]
|
||||
for offset in offsets:
|
||||
draw.text((text_x + offset[0], text_y_offset + offset[1]), line, fill=(255, 255, 255),
|
||||
font=font)
|
||||
|
||||
# 绘制文字
|
||||
draw.text((text_x, text_y_offset), line, fill=(0, 0, 0), font=font)
|
||||
|
||||
# 保存结果
|
||||
canvas.save(output_path)
|
||||
return output_path
|
||||
except Exception as e:
|
||||
logger.exception(f"拼图错误 {e}")
|
||||
return None
|
||||
|
||||
def upload(file_path, google_api_key):
|
||||
with open(file_path, "rb") as image:
|
||||
image = image.read()
|
||||
content_length = len(image)
|
||||
if file_path.split('.')[-1] == "jpg":
|
||||
content_type = f"image/jpeg"
|
||||
elif file_path.split('.')[-1] == "png":
|
||||
content_type = f"image/png"
|
||||
elif file_path.split('.')[-1] == "gif":
|
||||
content_type = f"image/gif"
|
||||
elif file_path.split('.')[-1] == "webp":
|
||||
content_type = f"image/webp"
|
||||
else:
|
||||
raise Exception(f"不支持的文件格式{file_path.split('.')[-1]}")
|
||||
filename = file_path.split("\\")[-1]
|
||||
logger.info(f"Uploading name = {filename}, size = {content_length}, type = {content_type} to google file")
|
||||
with httpx.Client(timeout=1800) as client:
|
||||
pre_upload_response = client.post(
|
||||
url=f"https://generativelanguage.googleapis.com/upload/v1beta/files?key={google_api_key}",
|
||||
headers={
|
||||
"X-Goog-Upload-Protocol": "resumable",
|
||||
"X-Goog-Upload-Command": "start",
|
||||
"X-Goog-Upload-Header-Content-Length": str(content_length),
|
||||
"X-Goog-Upload-Header-Content-Type": content_type
|
||||
},
|
||||
json={
|
||||
"file": {
|
||||
"display_name": filename.split(".")[0],
|
||||
}
|
||||
})
|
||||
pre_upload_response.raise_for_status()
|
||||
|
||||
upload_url = pre_upload_response.headers.get("X-Goog-Upload-Url")
|
||||
|
||||
upload_response = client.post(url=upload_url, content=image, headers={
|
||||
"X-Goog-Upload-Offset": "0",
|
||||
"X-Goog-Upload-Command": "upload, finalize",
|
||||
"Content-Type": content_type
|
||||
})
|
||||
upload_response.raise_for_status()
|
||||
|
||||
return upload_response.json(), upload_response.status_code
|
||||
|
||||
@SentryUtils.sentry_tracker(sentry_trace.x_trace_id, sentry_trace.x_baggage, op="make_grid_gemini", name="将输入图拼为网格上传到Gemini网盘", fn_id=current_function_call_id())
|
||||
def _handler(google_api_key:str,
|
||||
pic_info_list:List[Dict[str,str]],
|
||||
image_size:int,
|
||||
text_height:int,
|
||||
font_size:int,
|
||||
padding:int,
|
||||
separator:int):
|
||||
image_grid_path = f"grid_{uuid.uuid4()}.jpg"
|
||||
image_grid_path = create_image_grid(pic_info_list, image_grid_path, image_size, text_height, font_size, padding, separator)
|
||||
if not image_grid_path:
|
||||
raise Exception("创建图片网格失败")
|
||||
|
||||
image_grid_gemini, code = upload(image_grid_path, google_api_key)
|
||||
if code == 200:
|
||||
image_gemini_uri = image_grid_gemini["file"]["uri"]
|
||||
else:
|
||||
logger.error("图片网格文件上传Gemini失败")
|
||||
raise Exception("图片网格文件上传Gemini失败")
|
||||
return image_gemini_uri
|
||||
return _handler(google_api_key, pic_info_list, image_size, text_height, font_size, padding, separator)
|
||||
|
||||
|
||||
@app.function(max_containers=config.video_downloader_concurrency, timeout=130)
|
||||
@modal.concurrent(max_inputs=50)
|
||||
|
||||
Reference in New Issue
Block a user