Dolly is a large language model developed by Databricks and publicly released in 2023. Its main distinguishing feature is that it is an open source model, released under a permissive license and accompanied by detailed information about the training dataset and the fine-tuning process. In a landscape dominated by proprietary models accessible only through external interfaces, Dolly was a concrete example of how organizations not exclusively focused on artificial intelligence research can also produce a working, reusable language model.
The name references the cloned sheep Dolly, a playful nod to the idea of a model “cloned” from existing capabilities and made available to the community. Technically, Dolly is based on a transformer architecture and a supervised fine-tuning approach applied to an already pre-trained base model. This glossary entry describes what Dolly is, how it works, how it differs from other models, and what its main limitations are.
What sets an instruction-tuned model apart
Dolly is a large language model designed to follow instructions in natural language. Unlike base models, which simply complete likely text, an instruction-tuned model is trained to respond to requests phrased as commands, questions, or specific tasks. Dolly was made publicly available along with the documentation needed to reproduce its training, which sets it apart from many closed competitors.
Its importance in the SEO context and in technology education stems mainly from transparency: anyone can study how it works, adapt it to their own use cases, and verify its performance on real tasks. This makes it a useful reference for understanding what “open model” means today in natural language processing.
The two phases of training
Dolly follows a two-phase paradigm. In the first phase, a base model is pre-trained on enormous amounts of text to learn language structures, vocabulary, and general knowledge. In the second phase, called supervised fine-tuning, the model is exposed to pairs of instructions and correct responses, so it learns to behave like an assistant that performs tasks.
Transformer architecture
Dolly’s architecture is the transformer, the same family of models that revolutionized natural language processing. The transformer uses attention mechanisms to weigh the relative importance of words within a sentence and a larger text. This structure allows it to handle long contexts and complex relationships between parts of speech.
Supervised fine-tuning
Supervised fine-tuning is the step that turns a base model into a model capable of following instructions. The dataset used for Dolly was built with human contributions and made public, allowing other researchers to replicate the experiment. This approach demonstrated that a model of modest size, if trained with quality data, can achieve competitive results on many practical tasks.
Dolly compared to common alternatives
To place Dolly in the landscape of language models, it helps to compare it with some common alternatives. The following table summarizes the main differences in aspects such as openness, size, and intended use.
| Model | Openness | Typical size | Main use |
|---|---|---|---|
| Dolly | Open source | Parameters in the billions | Experimentation and research |
| Large proprietary models | Closed | Hundreds of billions | Commercial services |
| Small open models | Open source | Less than one billion | Local devices |
| Code-specialized models | Varies | Varies | Programming |
| Open multilingual models | Open source | Varies | Translation and analysis |
Where it can be used and what it cannot do
Dolly can be used in several scenarios: text generation, summarization, question answering, content classification, and writing assistance. In the SEO context, a model like this can support drafting, keyword analysis, and content structuring, always with human oversight. However, like all language models, it is not immune to errors and should not be considered an autonomous source of truth.
Strengths
The main advantages of Dolly are the transparency of the training process, the ability to run it on your own infrastructure, and the freedom to adapt it to specific domains. These elements make it interesting for universities, companies, and developers who want to understand and control the technology they use.
Limitations and risks
Limitations include the tendency to generate unverified information, the possibility of reproducing biases present in the training data, and world knowledge that is not updated beyond the data collection date. Moreover, performance on very complex tasks remains lower than that of larger proprietary models.
A path to evaluate it before adopting it
Anyone who wants to use Dolly in a project should follow a methodical path, from defining objectives to verifying results. The following steps summarize a reasonable approach.
- Clearly define the task the model must perform and the success criteria.
- Check the hardware requirements and the availability of adequate computing resources.
- Download the model and the fine-tuning dataset from official sources.
- Adapt the model to your domain with representative, high-quality data.
- Evaluate performance on a separate test set not used in training.
- Integrate human checks to correct errors and reduce the risk of incorrect content.
- Document the process and the system’s known limitations.
- Periodically update the model or data to maintain relevance.
Frequently asked questions about Dolly
What does it mean that Dolly is open source?
It means that the model and the materials needed to train it are made publicly available under a license that allows use, study, and modification. This promotes transparency and reproducibility of research.
Is Dolly a large model?
It falls into the category of large language models, but with fewer parameters than the most well-known proprietary models. Its relevance comes more from its openness than from its absolute size.
What are the differences compared to closed models?
Closed models do not make weights, data, and training details public, while Dolly provides much of this information. This allows results to be replicated and the model to be adapted to specific needs.
Does it require a large infrastructure to use?
It depends on the context: inference can require considerable resources, while full training needs dedicated hardware. However, reduced configurations exist for experimentation.
Can Dolly be used for SEO?
It can support activities such as drafting, summarization, and content analysis, but results should always be verified by a person. It does not replace editorial and strategic expertise.
What are the main risks of using Dolly?
Risks include the production of incorrect information, the reproduction of biases present in the data, and misuse of generated content. Careful human oversight significantly reduces these dangers.
From glossary to projectDo language models cite your site as a source?Optimizing for language models (LLMs) is about getting your brand cited in Google AI Overview answers and in the main LLMs.Discover LLM optimization
Related entries:
Open source artificial intelligence Generative pre-trained transformer Mistral 7B Llm Mixtral 8x7B