跳到正文
Rohan Paul· @rohanpaul_ai · X·· 2 小时前AI 评分62
AI 导读

Google 发布了 EmbeddingGemma 2 论文,推出面向端侧的开源多模态嵌入模型。该模型采用 Apache 2.0 协议开源,参数量为 744M,能将文本、代码、图像、音频和视频映射到同一个可搜索向量空间。在 Pixel 11 Pro 上,量化后的完整多模态模型约占用 567MB 内存,上下文窗口扩展至 8K tokens。

正文 · 原文

Google released its EmbeddingGemma 2 paper.

embedding model for on-device multimodal AI, under an Apache 2.0 license.

puts text, code, images, audio and video into 1 searchable space on phones. gives phones a missing piece: a way to understand and search your own stuff without sending it to a server.

shows that a 744M-parameter open model can embed text, images, audio, and video, beating many models 2 to 12 times larger.

On a Pixel 11 Pro, quantized text weights take about 191MB of active RAM, and the full multimodal model takes about 567MB.

The context window grows 4x to 8K tokens, enough for roughly 5.5 minutes of audio, 29 images or 58 video frames in 1 input.

They built EmbeddingGemma 2 on Gemma 4, trained it to copy the larger Gemini Embedding 2, then tuned it with contrastive training. All media types share 1 vector space, and the full model fits in 567 MB on a Pixel 11 Pro with 4-bit weights.

Embedding speech directly beat transcribing it, 69.54 versus 57.59 on the MSEB spoken-query retrieval benchmark. Code search scored 78.68 on MTEB Code, up from 68.76 for EmbeddingGemma 1.

来源:Rohan Paul · x.com