Agent在多模态任务中如何执行推理？

1.基本思路

import openai import base64 # 读取图片并转为 base64 with open("image.jpg", "rb") as f: image_data = base64.standard_b64encode(f.read()).decode("utf-8") response = openai.chat.completions.create( model="gpt-4-vision-preview", messages=[{ "role": "user", "content": [ {"type": "text", "text": "这张图里有什么？"}, {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}} ] }] )

2.跨膜态对齐的底层原理

3.BLIP-2的轻量级思路

4.视频的时序推理

5.工程实践中的注意点

6.追问