Azercell and AWS Develop Azerbaijani LLM on SageMaker AI
TL;DR. Azercell collaborated with the AWS Generative AI Innovation Center to build an Azerbaijani large language model on Amazon SageMaker AI. - The project focused on adapting foundation models for morphologically rich languages with limited data. - It achieved a 23% higher training throughput and 58% lower peak GPU memory usage. - A custom tokenizer also delivered a 2x improvement in tokens per word for Azerbaijani.
- Azercell developed an Azerbaijani LLM on Amazon SageMaker AI for telecom use cases and chatbots.
- The process addressed challenges of adapting foundation models to morphologically rich languages.
- Collaboration with AWS resulted in a production-ready framework with significant performance gains.
- Optimizations included kernel-level improvements and a custom tokenizer for Azerbaijani.
Sources
- Training Azerbaijani language models on Amazon SageMaker AI — aws.amazon.com