At some point you need to move glyphs from CPU to GPU. All of the usual optimization tips apply here.

Thanks, the cache suggestion works perfect.