Custom Multimodal Language Model (MLM) processing up to 3 images + text with Rotary Position Embeddings (RoPE), 2D residual patch projections, SwiGLU, and dynamic entropy-driven adaptive Top-K MoE.
SwiftFox (Multimodal MoE)
Custom Multimodal Language Model (MLM) processing up to 3 images + text with Rotary Position Embeddings (RoPE), 2D residual patch projections, SwiGLU, and dynamic entropy-driven adaptive Top-K MoE.
