Querysets

QuerySets and managers, and the relationship between them, are some of the most difficult concepts to understand for new Django developers. I'm not going to attempt to explain the functional difference here (there's plenty online about that) but instead make a conceptual observation. It boils down to this: querysets are state containers and managers are code containers.

A queryset is a tool for incrementally building an SQL query. It has some internal state that represents the query being built, and exposes a set of methods for constructing, manipulating and mutating that state (the internal state is actually cloned, rather than mutated, but that's not relevant for this discussion). The queryset abstraction is designed around a pattern called a fluent interface, where each of the state-modifying methods returns (a copy of) the state container (self), allowing query methods to be chained in the familiar manner. Querysets are, in my opinion, one of the most beautiful and powerful concepts in Django, and true mastery of Django depends largely on developing a deep understanding of querysets.

A manager, on the other hand, is a bit of a confused concept, and the confusion is rooted in the design of the ORM. A model class in Django is both a declaration of the structure of a database table and a container for row-level (instance-level) behaviour, implemented in most codebases as methods on the model class. This leaves nowhere obvious to put behaviour related to the table. It could be argued that the entire manager concept could be replaced with @classmethod-decorated methods on the model, but having a separate namespace in which to group these table-level methods is, on balance, probably helpful. So that's what a manager is: a namespace to group together table-level functionality. It doesn't have any user-visible state of its own.

The confusion is compounded because many of the methods from the queryset are duplicated (in early versions of Django they were literally copied-and-pasted!) onto the manager implementation. When you call SomeModel.objects.filter(...).filter(...), the first .filter is actually being called on the manager instance (the objects attribute). This .filter method returns a queryset, so the second (and any subsequent) .filter calls are being made on the queryset. A perfect example of a well-intentioned decision to optimise for conciseness in API design creating cognitive load in generations of developers for decades afterwards. Explicit is better than implicit!

So, with the conceptual discussion out of the way, let's turn to practical matters. If we want to add business logic to the ORM, where should it go?

I have huge respect for James Bennett. There are few people who've done more to contribute to Django than James, and I largely agree with almost everything he says about abstraction, encapsulation and API design. However, it's worth quoting his explanation of this at length, to try to illustrate how difficult these concepts are to understand. The below extended quote is from his (brilliant) essay Against service layers in Django, and I've edited some of the points slightly for brevity.

Quote

  • If you need an alternate constructor for model instances, do it as a manager method (not as a classmethod on the model, or in some other class or code elsewhere) [...]
  • If you have a custom, complex and/or often-used query against the model's entire table, do it as either a manager method (if you think you’ll never need to use it chained after other query methods), or as a QuerySet method (if you will need to be able to chain it after other things) [...]
  • If you need more complex fetching of related objects from a single model instance, do it as a method on that model class.
  • Any sort of logical operation on only a single model instance should be a method on the model class.
  • Any sort of logical operation on a set of instances of the same model should be a method on the model’s manager or its QuerySet.

— James Bennett

James introduces the above bullets as "a quick set of recommendations for how to implement things"! It's really worth reading the essay in detail for the full weight of complexity he's attempting to convey. His argument is that there's a set of straightforward rules for deciding where to put business logic, but I rather feel he's making exactly the opposite point.

The thing we agree on here is that a "service layer" is not the answer. It doesn't help anyone to hide Django under yet another layer of complexity: what we want is an easier-to-understand answer to these questions.

We'll get to that in the next section.

Summary

The question of where to put table-level or query-level business logic in a Django application is difficult and confusing to answer, especially for new developers, and suffers from many of the same problems as instance-level business logic.