comi-logic/modalDeploy
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

合并分支

Browse Source 
* FIX
* FIX 适配图像质量选项 FIX 修复fs参数
* Merge branch 'main' into feature/modal-cluster-gemini
* Merge branch 'main' into feature/modal-cluster-gemini
* ADD 增加大模型输出错误时间值的处理
* Merge branch 'main' into feature/modal-cluster-gemini
* FIX 修复推理问题
* Merge branch 'main' into feature/modal-cluster-gemini
* FIX 修复拼接网格图的异步失效问题
* FIX 修复拼接网格图的异步失效问题
* Merge branch 'main' into feature/modal-cluster-gemini
* Merge branch 'main' into feature/modal-cluster-gemini
* FIX 调高合成网格图容器配置
* Merge branch 'main' into feature/modal-cluster-gemini
* FIX 修复带图片模态推理接口
* PERF 优化Gemini Prompt词
* Merge remote-tracking branch 'origin/main' into feature/modal-cluster-gemini
* ADD 增加网格图片创建接口 ADD gemini推理打点增加商品封面模态 PERF gemini打点结果相邻时间线同商品合并
* Merge branch 'main' into feature/modal-cluster-gemini
* FIX 429重试机制
* FIX 修复Gemini推理函数入参
* Merge branch 'main' into feature/modal-cluster-gemini
* PERF gemini修改参数类型 ADD 增加批量删除Gemini文件接口
* PERF gemini修改prompt
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* PERF gemini修改prompt
* 录制HLS时 结果复制到S3挂载点增加一个fallback处理
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* PERF gemini修改prompt
* PERF gemini增加图像模态
* PERF gemini增加图像模态
* fix : TikHub的Logger换为Loguru
* fix : 修复直播切分视频接口
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* PERF tikhub接口增加随机延迟
* Merge remote-tracking branch 'origin/feature/modal-cluster' into feature/modal-cluster
* 合并分支
* fix : hls录制缓存读写问题
* FIX 修改冲突
* ADD 增加获取直播间指定商品数量接口
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* PERF 修改tikhub到aws
* fix : 增加s3分片上传接口
* PERF Gemini推理直接从参数列表获取商品列表 ADD Tikhub增加获取置顶|讲解中商品接口
* PERF Gemini推理直接从参数获取商品列表
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* fix 处理jpg图片的codec_name应为mjpg
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* PERF 优化返回值
* fix 获取媒体metadata的类型报错，modal client升级到1.0.2
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* PERF 风控后爬虫直接退出避免cookie失效 PERF 分离tikhub api, 使用更低的保持时间以更换IP FIX 修复Dict键不存在报错的bug
* fix 一些webhook相关的bug，添加了更详细的接口说明
* 更新直播录制为可跳转时间的hls流接口，用于替代掉腾讯VOD+火山云拉流转推，片段保证每片开头为关键帧，时间精度准确到毫秒级
* ADD 增加直播间商品增量缓存触发器
* Merge branch 'feature/modal-cluster' into feature/modal-cluster-gemini
* PERF 调整gemini prompt
* - KVCache类改为可拓展，基于环境变量设置KV space - test 环境配置与CF测试环境对齐
* FIX 修复Gemini推理视频切条的问题 PERF 优化视频切条函数性能
* ADD 基本完成gemini视频推理API
* ADD 添加内联的tikhub
* 对齐预发环境

---------

Merge request URL: https://g-ldyi2063.coding.net/p/dev/d/modalDeploy/git/merge/4804
Co-authored-by: 康宇佳，shuohigh@gmail.com
This commit is contained in:
康宇佳
2025-06-13 18:17:18 +08:00
committed by Coding
parent 91ebad57c6
commit c4d596a31c
5 changed files with 27 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
src/BowongModalFunctions/models/web_model.py
View File

@@ -442,6 +442,7 @@ class GeminiRequest(BaseFFMPEGTaskRequest):
product_list: List[Union[str, dict]] = Field(description="商品名列表(时间倒序)"),
start_time: str = Field(default="00:00:00.000", description="开始时间(hls)")
end_time: str = Field(default="00:20:00.000", description="结束时间(hls)")
options: FFMPEGSliceOptions = Field(default=FFMPEGSliceOptions() ,description="输出质量选项")
@field_validator('media_hls_url', mode='before')
@classmethod

2
src/BowongModalFunctions/router/google.py
View File

@@ -152,7 +152,7 @@ async def inference_gemini(
if not google_api_key:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Missing Google API Key")
fn = modal.Function.from_name(config.modal_app_name,"video_hls_slice_inference", environment_name=config.modal_environment)
fn_call = fn.spawn(data.media_hls_url, google_api_key, data.product_cover_grid_uri_list, data.product_list, data.start_time, data.end_time,
fn_call = fn.spawn(data.media_hls_url, google_api_key, data.product_cover_grid_uri_list, data.product_list, data.start_time, data.end_time, data.options,
SentryTransactionInfo(x_trace_id=sentry_sdk.get_traceparent(), x_baggage=sentry_sdk.get_baggage())
if headers.x_trace_id is None else SentryTransactionInfo(x_trace_id=headers.x_trace_id, x_baggage=headers.x_baggage))
return ModalTaskResponse(success=True, taskId=fn_call.object_id)

4
src/BowongModalFunctions/utils/VideoUtils.py
View File

@@ -51,9 +51,9 @@ class FFMPEGSliceOptions(BaseModel):
if self.limit_size is not None:
for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
if self.limit_size < 1024.0:
return f"{self.limit_size:.2f} {unit}"
return f"{self.limit_size:.2f}{unit}"
self.limit_size /= 1024.0
return f"{self.limit_size:.2f} PB"
return f"{self.limit_size:.2f}PB"
else:
return None

26
src/cluster/video_apps/hls_slice_inference.py
View File

@@ -19,6 +19,7 @@ with downloader_image.imports():
from BowongModalFunctions.models.web_model import SentryTransactionInfo, WebhookNotify
from BowongModalFunctions.models.ffmpeg_worker_model import FFMpegSliceSegment
from BowongModalFunctions.utils.TimeUtils import TimeDelta, merge_product_data
from BowongModalFunctions.utils.VideoUtils import FFMPEGSliceOptions
@app.function(cpu=(0.5, 64), timeout=1800,
@@ -28,7 +29,9 @@ with downloader_image.imports():
bucket_name=config.S3_bucket_name,
secret=modal.Secret.from_name("aws-s3-secret", environment_name=config.modal_environment),
),
})
},
region="us"
)
@modal.concurrent(max_inputs=1)
async def video_hls_slice_inference(media: MediaSource,
google_api_key: str,
@@ -36,11 +39,13 @@ with downloader_image.imports():
product_list: List[any],
start_time: str,
end_time: str,
options: FFMPEGSliceOptions,
sentry_trace: SentryTransactionInfo,
webhook: WebhookNotify = None,
retry_time: int = 3
):
logger.info(
f"region {os.environ.get('MODAL_REGION', 'unknown')}, provider {os.environ.get('MODAL_CLOUD_PROVIDER', 'unknown')}")
def calculate_target_dimensions(original_width, original_height, target_height=1280):
"""计算保持宽高比的目标尺寸，高度固定为1280"""
aspect_ratio = original_width / original_height
@@ -177,15 +182,16 @@ with downloader_image.imports():
slice_fn = modal.Function.from_name(config.modal_app_name, "ffmpeg_slice_media",
environment_name=config.modal_environment)
slice_result, sentry_trace = await slice_fn.remote.aio(media, [FFMpegSliceSegment(
start=TimeDelta.from_format_string(start_time), end=TimeDelta.from_format_string(end_time))],
start=TimeDelta.from_format_string(start_time), end=TimeDelta.from_format_string(end_time))], options,
sentry_trace)
video = MediaSource.from_str(slice_result[0].urn)
logger.success("截取完成")
# 3、转换为720p h265
logger.info("3、转换视频为720p h264")
video_path = convert_video(os.path.join(config.S3_mount_dir, video.path))
# logger.info("3、转换视频为720p h264")
# video_path = convert_video(os.path.join(config.S3_mount_dir, video.path))
video_path = os.path.join(config.S3_mount_dir, video.path)
# 4、上传到gemini
logger.info("4、视频文件开始上传到Gemini")
logger.info("3、视频文件开始上传到Gemini")
video_gemini, code = upload(video_path)
if code == 200:
video_gemini_uri = video_gemini["file"]["uri"]
@@ -206,7 +212,7 @@ with downloader_image.imports():
return False
check_num = 60
logger.info("5、开始检查Gemini文件是否处理完成")
logger.info("4、开始检查Gemini文件是否处理完成")
while check_num > 0:
logger.info(f"检查中, 剩余检查次数{check_num}")
ret = check()
@@ -383,7 +389,7 @@ with downloader_image.imports():
await asyncio.sleep(random.randint(50, 70))
return None
logger.info("6、发起Gemini推理")
logger.info("5、发起Gemini推理")
target_json = None
while target_json is None and retry_time > 0:
target_json = await inference_api()
@@ -398,7 +404,7 @@ with downloader_image.imports():
parts = parts.replace("```", "").replace("json\n", "").replace("\n", "").replace("\\", "")
parts = json.loads(parts)
# 合并产品和时间线
parts = merge_product_data(parts)
parts = merge_product_data(parts, start_time, end_time)
return parts, sentry_trace
else:
raise Exception(f"Gemini推理失败, 失败原因: {reason}")
@@ -408,7 +414,7 @@ with downloader_image.imports():
raise Exception(f"推理失败, {e}")
finally:
if video_gemini_uri:
logger.info("7、清除Gemini临时文件")
logger.info("6、清除Gemini临时文件")
with httpx.Client(timeout=Timeout(timeout=120)) as client:
resp = client.delete(
f'https://bowongai-{config.modal_environment}--{config.modal_app_name}-fastapi-webapp.modal.run/google/delete',

8
src/cluster/video_apps/make_grid_upload.py
View File

@@ -15,7 +15,11 @@ with downloader_image.imports():
from BowongModalFunctions.utils.SentryUtils import SentryUtils
from BowongModalFunctions.models.web_model import SentryTransactionInfo
@app.function(cpu=(0.5, 16), max_containers=config.video_downloader_concurrency, timeout=240)
@app.function(cpu=(0.5, 16),
max_containers=config.video_downloader_concurrency,
timeout=240,
region="ap",
)
@modal.concurrent(max_inputs=200)
async def make_image_grid_upload(pic_info_list: List[Dict[str, str]],
image_size: int,
@@ -25,6 +29,8 @@ with downloader_image.imports():
separator: int,
google_api_key: str,
sentry_trace: SentryTransactionInfo) -> str:
logger.info(
f"region {os.environ.get('MODAL_REGION', 'unknown')}, provider {os.environ.get('MODAL_CLOUD_PROVIDER', 'unknown')}")
async def create_image_grid(image_info_list: List[Dict[str, str]],
output_path: str,
image_size: int = 450,