In a significant development in the field of Artificial Intelligence, Apple has launched OpenELM, a new set of AI language models that are small enough to operate directly on mobile devices like smartphones.
OpenELM, which stands for “Open-source Efficient Language Models,” represents a shift towards more accessible AI capabilities that do not require the power of cloud-based data centers.
As of now, these models are primarily experimental but they lay the groundwork for potential widespread use in Apple’s future products.
Critics seem to still be excited over Apple’s new invention:
Comment
byu/Tooskee from discussion
insingularity
OpenELM is currently available on Hugging Face under an Apple Sample Code License, which, while having some restrictions, still offers public access to the source code.
This initiative is part of a broader trend where companies are aiming to integrate more powerful AI capabilities directly into handheld devices.
Apple’s OpenELM consists of eight different models, with capacities ranging from 270 million to 3 billion parameters. This variety offers several levels of complexity and efficiency, tailored to different types of AI applications.
Apple’s advancement in the world of AI has a buzz on Twitter as well:
Want smaller AI models that run on your phone? 📱
Apple just released OpenELM, open-source language models ranging from 270M to 3B parameters. 🤖 #Read4U #AppleAI #OnDeviceAI
What do you think? 🤔https://t.co/3beT8MZmcd— Aili (@aili_app) April 26, 2024
The models are available in two formats: four “pre-trained” models that provide basic AI functions, and four “instruction-tuned” models, which are optimized for tasks like operating AI assistants and chatbots.
OpenELM operates with a maximum context window of 2048 tokens, which are small units of data processed by AI language models.
These models have been trained on several datasets including RefinedWeb and subsets of RedPajama and Dolma v1.6, totaling approximately 1.8 trillion tokens.
Apple’s innovative “layer-wise scaling strategy” in these models optimizes parameter use across each layer of the AI, enhancing performance while requiring fewer computational resources.
According to a white paper released by Apple, this strategy has improved the accuracy of these models by 2.36 percent compared to Allen AI’s OLMo 1B model, and it has achieved this with half the number of pre-training tokens typically required.
Here’s what a twitter user have got to say about this:
Apple’s recent OpenELM is very interesting and showcases that a layer-wise scaling technique can be more optimal, at least for small language model regime. Some insights:
– Adopts Transformer modifications from the DeLight paper: starts with narrower, then wider blocks… pic.twitter.com/Re9HFAPI6p
— Amir Gholami (@amir__gholami) April 26, 2024
While these models are a promising advance in AI technology, Apple has noted that they may produce outputs that could be inaccurate or biased due to their training on publicly available data. The company advises caution in their application.
Looking forward, it is rumored that the next iOS update might incorporate these new AI features, enhancing on-device processing to improve user privacy.
This could potentially elevate the capabilities of Apple’s devices significantly, impacting how consumers interact with their smartphones and other Apple devices in the future.
To find out more for the latest and most exciting AI News, visit www.allaboutai.com.
