Home

Awesome

Django REST - FlexFields

Package version Python versions

Flexible, dynamic fields and nested models for Django REST Framework serializers.

Overview

FlexFields (DRF-FF) for Django REST Framework is a package designed to provide a common baseline of functionality for dynamically setting fields and nested models within DRF serializers. This package is designed for simplicity, with minimal magic and entanglement with DRF's foundational classes.

Key benefits:

Quick Start

from rest_flex_fields import FlexFieldsModelSerializer

class StateSerializer(FlexFieldsModelSerializer):
    class Meta:
        model = State
        fields = ('id', 'name')

class CountrySerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Country
        fields = ('id', 'name', 'population', 'states')
        expandable_fields = {
          'states': (StateSerializer, {'many': True})
        }

class PersonSerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Person
        fields = ('id', 'name', 'country', 'occupation')
        expandable_fields = {'country': CountrySerializer}

GET /people/142/

{
  "id": 142,
  "name": "Jim Halpert",
  "country": 1
}

GET /people/142/?expand=country.states

{
  "id": 142,
  "name": "Jim Halpert",
  "country": {
    "id": 1,
    "name": "United States",
    "states": [
      {
        "id": 23,
        "name": "Ohio"
      },
      {
        "id": 2,
        "name": "Pennsylvania"
      }
    ]
  }
}

Table of Contents:

Setup

First install:

pip install drf-flex-fields

Then have your serializers subclass FlexFieldsModelSerializer:

from rest_flex_fields import FlexFieldsModelSerializer

class StateSerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Country
        fields = ('id', 'name')

class CountrySerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Country
        fields = ('id', 'name', 'population', 'states')
        expandable_fields = {
          'states': (StateSerializer, {'many': True})
        }

Alternatively, you can add the FlexFieldsSerializerMixin mixin to a model serializer.

Usage

Dynamic Field Expansion

To define expandable fields, add an expandable_fields dictionary to your serializer's Meta class. Key the dictionary with the name of the field that you want to dynamically expand, and set its value to either the expanded serializer or a tuple where the first element is the serializer and the second is a dictionary of options that will be used to instantiate the serializer.

class CountrySerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Country
        fields = ['name', 'population']


class PersonSerializer(FlexFieldsModelSerializer):
    country = serializers.PrimaryKeyRelatedField(read_only=True)

    class Meta:
        model = Person
        fields = ['id', 'name', 'country', 'occupation']

        expandable_fields = {
            'country': CountrySerializer
        }

If the default serialized response is the following:

{
  "id": 13322,
  "name": "John Doe",
  "country": 12,
  "occupation": "Programmer"
}

When you do a GET /person/13322?expand=country, the response will change to:

{
  "id": 13322,
  "name": "John Doe",
  "country": {
    "name": "United States",
    "population": 330000000
  },
  "occupation": "Programmer"
}

Deferred Fields

Alternatively, you could treat country as a "deferred" field by not defining it among the default fields. To make a field deferred, only define it within the serializer's expandable_fields.

Deep, Nested Expansion

Let's say you add StateSerializer as a serializer nested inside the country serializer above:

class StateSerializer(FlexFieldsModelSerializer):
    class Meta:
        model = State
        fields = ['name', 'population']


class CountrySerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Country
        fields = ['name', 'population']

        expandable_fields = {
            'states': (StateSerializer, {'many': True})
        }

class PersonSerializer(FlexFieldsModelSerializer):
    country = serializers.PrimaryKeyRelatedField(read_only=True)

    class Meta:
        model = Person
        fields = ['id', 'name', 'country', 'occupation']

        expandable_fields = {
            'country': CountrySerializer
        }

Your default serialized response might be the following for person and country, respectively:

{
  "id" : 13322,
  "name" : "John Doe",
  "country" : 12,
  "occupation" : "Programmer",
}

{
  "id" : 12,
  "name" : "United States",
  "states" : "http://www.api.com/countries/12/states"
}

But if you do a GET /person/13322?expand=country.states, it would be:

{
  "id": 13322,
  "name": "John Doe",
  "occupation": "Programmer",
  "country": {
    "id": 12,
    "name": "United States",
    "states": [
      {
        "name": "Ohio",
        "population": 11000000
      }
    ]
  }
}

Please be kind to your database, as this could incur many additional queries. Though, you can mitigate this impact through judicious use of prefetch_related and select_related when defining the queryset for your viewset.

Field Expansion on "List" Views <a id="list-views"></a>

If you request many objects, expanding fields could lead to many additional database queries. Subclass FlexFieldsModelViewSet if you want to prevent expanding fields by default when calling a ViewSet's list method. Place those fields that you would like to expand in a permit_list_expands property on the ViewSet:

from rest_flex_fields import is_expanded

class PersonViewSet(FlexFieldsModelViewSet):
    permit_list_expands = ['employer']
    serializer_class = PersonSerializer

    def get_queryset(self):
        queryset = models.Person.objects.all()
        if is_expanded(self.request, 'employer'):
            queryset = queryset.select_related('employer')
        return queryset

Notice how this example is using the is_expanded utility method as well as select_related and prefetch_related to efficiently query the database if the field is expanded.

Expanding a "Many" Relationship <a id="expanding-many"></a>

Set many to True in the serializer options to make sure "to many" fields are expanded correctly.

class StateSerializer(FlexFieldsModelSerializer):
    class Meta:
        model = State
        fields = ['name', 'population']


class CountrySerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Country
        fields = ['name', 'population']

        expandable_fields = {
            'states': (StateSerializer, {'many': True})
        }

A request to GET /countries?expand=states will return:

{
    "id" : 12,
    "name" : "United States",
    "states" : [
      {
        "name" : "Alabama",
        "population": 11000000
      },
      //... more states ... //
      {
        "name" : "Ohio",
        "population": 11000000
      }
    ]
}

Dynamically Setting Fields (Sparse Fields) <a id="dynamically-setting-fields"></a>

You can use either the fields or omit keywords to declare only the fields you want to include or to specify fields that should be excluded.

Consider this as a default serialized response:

{
  "id": 13322,
  "name": "John Doe",
  "country": {
    "name": "United States",
    "population": 330000000
  },
  "occupation": "Programmer",
  "hobbies": ["rock climbing", "sipping coffee"]
}

To whittle down the fields via URL parameters, simply add ?fields=id,name,country to your requests to get back:

{
  "id": 13322,
  "name": "John Doe",
  "country": {
    "name": "United States",
    "population": 330000000
  }
}

Or, for more specificity, you can use dot-notation, ?fields=id,name,country.name:

{
  "id": 13322,
  "name": "John Doe",
  "country": {
    "name": "United States"
  }
}

Or, if you want to leave out the nested country object, do ?omit=country:

{
  "id": 13322,
  "name": "John Doe",
  "occupation": "Programmer",
  "hobbies": ["rock climbing", "sipping coffee"]
}

Reference serializer as a string (lazy evaluation) <a id="lazy-ref"></a>

To avoid circular import problems, it's possible to lazily evaluate a string reference to you serializer class using this syntax:

expandable_fields = {
    'record_set': ('<module_path_to_serializer_class>.RelatedSerializer', {'many': True})
}

Note: Prior to version 0.9.0, it was assumed your serializer classes would be in a module with the following path: <app_name>.serializers.

This import style will still work, but you can also now specify fully-qualified import paths to any locations.

Increased re-usability of serializers <a id="increased-reuse"></a>

The omit and fields options can be passed directly to serializers. Rather than defining a separate, slimmer version of a regular serializer, you can re-use the same serializer and declare which fields you want.

from rest_flex_fields import FlexFieldsModelSerializer

class CountrySerializer(FlexFieldsModelSerializer):
    class Meta:
        model = Country
        fields = ['id', 'name', 'population', 'capital', 'square_miles']

class PersonSerializer(FlexFieldsModelSerializer):
    country = CountrySerializer(fields=['id', 'name'])

    class Meta:
        model = Person
        fields = ['id', 'name', 'country']


serializer = PersonSerializer(person)
print(serializer.data)

>>>{
  "id": 13322,
  "name": "John Doe",
  "country": {
    "id": 1,
    "name": "United States",
  }
}

Serializer Options

Dynamic field options can be passed in the following ways:

Approach #1

GET /people?expand=friends.hobbies,employer&omit=age

Approach #2

serializer = PersonSerializer(
  person,
  expand=["friends.hobbies", "employer"],
  omit="friends.age"
)

Approach #3


class PersonSerializer(FlexFieldsModelSerializer):
  // Your field definitions

  class Meta:
    model = Person
    fields = ["age", "hobbies", "name"]
    expandable_fields = {
      'friends': (
        'serializer.FriendSerializer',
        {'many': True, "expand": ["hobbies"], "omit": ["age"]}
      )
    }
OptionDescription
expandFields to expand; must be configured in the serializer's expandable_fields
fieldsFields that should be included; all others will be excluded
omitFields that should be excluded; all others will be included

Advanced

Customization

Parameter names and wildcard values can be configured within a Django setting, named REST_FLEX_FIELDS.

OptionDescriptionDefault
EXPAND_PARAMThe name of the parameter with the fields to be expanded"expand"
MAXIMUM_EXPANSION_DEPTHThe max allowed expansion depth. By default it's unlimited. Expanding state.towns would equal a depth of 2None
FIELDS_PARAMThe name of the parameter with the fields to be included (others will be omitted)"fields"
OMIT_PARAMThe name of the parameter with the fields to be omitted"omit"
RECURSIVE_EXPANSION_PERMITTEDIf False, an exception is raised when a recursive pattern is foundTrue
WILDCARD_VALUESList of values that stand in for all field names. Can be used with the fields and expand parameters. <br><br>When used with expand, a wildcard value will trigger the expansion of all expandable_fields at a given level.<br><br>When used with fields, all fields are included at a given level. For example, you could pass fields=name,state.* if you have a city resource with a nested state in order to expand only the city's name field and all of the state's fields. <br><br>To disable use of wildcards, set this setting to None.["*", "~all"]

For example, if you want your API to work a bit more like JSON API, you could do:

REST_FLEX_FIELDS = {"EXPAND_PARAM": "include"}

Defining Expansion and Recursive Limits on Serializer Classes

A maximum_expansion_depth integer property can be set on a serializer class.

recursive_expansion_permitted boolean property can be set on a serializer class.

Both settings raise serializers.ValidationError when conditions are met but exceptions can be customized by overriding the recursive_expansion_not_permitted and expansion_depth_exceeded methods.

Serializer Introspection

When using an instance of FlexFieldsModelSerializer, you can examine the property expanded_fields to discover which fields, if any, have been dynamically expanded.

Use of Wildcard to Match All Fields <a id="wildcards"></a>

You can pass expand=* (or another value of your choosing) to automatically expand all fields that are available for expansion at a given level. To refer to nested resources, you can use dot-notation. For example, requesting expand=menu.sections for a restaurant resource would expand its nested menu resource, as well as that menu's nested sections resource.

Or, when requesting sparse fields, you can pass fields=* to include only the specified fields at a given level. To refer to nested resources, you can use dot-notation. For example, if you have an order resource, you could request all of its fields as well as only two fields on its nested restaurant resource with the following: fields=*,restaurent.name,restaurant.address&expand=restaurant.

Combining Sparse Fields and Field Expansion <a id="combining-sparse-and-expanded"></a>

You may be wondering how things work if you use both the expand and fields option, and there is overlap. For example, your serialized person model may look like the following by default:

{
  "id": 13322,
  "name": "John Doe",
  "country": {
    "name": "United States"
  }
}

However, you make the following request HTTP GET /person/13322?include=id,name&expand=country. You will get the following back:

{
  "id": 13322,
  "name": "John Doe"
}

The fields parameter takes precedence over expand. That is, if a field is not among the set that is explicitly alllowed, it cannot be expanded. If such a conflict occurs, you will not pay for the extra database queries - the expanded field will be silently abandoned.

Utility Functions <a id="utils"></a>

rest_flex_fields.is_expanded(request, field: str)

Checks whether a field has been expanded via the request's query parameters.

Parameters

rest_flex_fields.is_included(request, field: str)

Checks whether a field has NOT been excluded via either the omit parameter or the fields parameter.

Parameters

Query optimization (experimental)

An experimental filter backend is available to help you automatically reduce the number of SQL queries and their transfer size. This feature has not been tested thorougly and any help testing and reporting bugs is greatly appreciated. You can add FlexFieldFilterBackend to DEFAULT_FILTER_BACKENDS in the settings:

# settings.py

REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': (
        'rest_flex_fields.filter_backends.FlexFieldsFilterBackend',
        # ...
    ),
    # ...
}

It will automatically call select_related and prefetch_related on the current QuerySet by determining which fields are needed from many-to-many and foreign key-related models. For sparse fields requests (?omit=fieldX,fieldY or ?fields=fieldX,fieldY), the backend will automatically call only(*field_names) using only the fields needed for serialization.

WARNING: The optimization currently works only for one nesting level.

Changelog <a id="changelog"></a>

1.0.2 (March 2023)

1.0.1 (March 2023)

1.0.0 (August 2022)

0.9.9 (July 2022)

0.9.8 (April 2022)

0.9.7 (January 2022)

0.9.6 (November 2021)

0.9.5 (October 2021)

0.9.3 (August 2021)

0.9.2 (June 2021)

0.9.1 (June 2021)

0.9.0 (April 2021)

0.8.9 (February 2021)

0.8.8 (September 2020)

0.8.6 (September 2020)

0.8.5 (May 2020)

0.8.1 (May 2020)

0.8.0 (April 2020)

0.7.5 (February 2020)

0.7.0 (February 2020)

0.6.1 (September 2019)

0.5.0 (April 2019)

0.3.4 (May 2018)

0.3.3 (April 2018)

Testing

Tests are found in a simplified DRF project in the /tests folder. Install the project requirements and do ./manage.py test to run them.

License

See License.