Always use gettext_lazy

I’ve been bitten many times by accidentally using gettext when I should have used gettext_lazy, resulting in strings that were stuck in English or Swedish randomly because a user with a specific language caused that piece of code to be imported.

I realize that there are some performance implications here, but compared to stuff like database access this is tiny and has never shown up in profiler outputs, so I will gladly take this hit and avoid these bugs that tend to be hard to track down (if they even get reported by users at all!).

A simple naive hand-rolled static analysis test that forbids usages of plain gettext in the code base is easy to implement and stops a whole class of bugs.

Django models

The Okrand setting django_model_upgrade which dynamically sets verbose_name for all fields correctly with the normal default, and on the model sets up verbose_name and verbose_name_plural. Then when you run the Okrand collect command you will get strings to translate without polluting your source with silly stuff like

class Foo(Model):
    user = ForeignKey(User, verbose_name=gettext_lazy('user'))
    
    class Meta:
        verbose_name = gettext_lazy('foo')
        verbose_name_plural = gettext_lazy('foos')

and you can instead have models like:

class Foo(Model):
    user = ForeignKey(User)

You can still write them out explicitly if you need them to differ from the defaults.

Elm

There’s a built-in regex pattern for ML-style languages in Okrand that makes it quite easy to collect strings from Elm code.

Menu translations

I use the iommi MainMenu system which looks something like this:

menu = MainMenu(
    items=dict(
        albums=M(view=albums_view),
        artists=M(view=artists_view),
    ),
)

Since Okrand has a plugin system, I can build a little function that loops over this menu and collects these identifiers into translation strings. In the example above this would be “albums” and “artists”. I enjoy not having to write the English base string that is 99% the exact same as the identifier (after replacing _ with space), which keeps the business logic clean.

Stick to lowercase as far as possible

I was frustrated by the translation files ending up with translations for “album” and “Album”, “artist” and “Artist” over and over. The solution I came up with was to define two simple functions:

def Trans(s):
    return capfirst(gettext_lazy(s))

def trans(s):
    return gettext_lazy(s)

I like the semantic weight of having Trans("album") mean that the word should start with uppercase in that place while trans("album") meaning that it should stay as lowercase. One could also add TRANS("album") if one wants all uppercase of a string for example.

High level APIs with steep learning curves (like iommi) are now just as easy to use as simpler APIs, since the cost of initial learning is moved from the human to the AI. Since we also invested heavily in great error messages and validating as much as possible up front, the feedback to the AI models is great. We’ve been banging the drum of “no silent failures!” for a decade, and nothing kills human or AI productivity as silent failures.

This is the time to focus our attention as humans to making APIs that are succinct and clear. It was vital before, but it’s growing in importance for every day.

Since I have a project with extensive documentation that I’ve spelled checked thoroughly this interested me. I write all the documentation with PyCharm which has built in spelling and grammar checks, so I was thinking it would be hard to find many errors.

I sent this prompt to Claude:

Go through the docs directory. Strings marked with # language: rst will be visible as normal text in the documentation. Suggest spelling, grammar, and language clarity improvements.

Claude fires up ~8 sub agents and found a surprising amount of things. Every single change was good.

A funny detail was that Claude ignored my request to only check the docs directory and found some issues in docstrings in the main source code. I can’t be angry about that :P

The funniest mistake was that the docs had the word “underling” instead of “underlying” in one place (“feature set of the underling Query and Form classes”). Perfectly fine spelling and grammar, but Claude correctly spots that this is mistake.

If you have some documentation, you definitely should give this a shot.

def test_grouped_fields():
    # language=rst
    """
    .. _group-fields:

    How do I group fields?
    ~~~~~~~~~~~~~~~~~~~~~~

    .. uses Field.group

    Use the `group` field:
    """

    form = Form(
        auto__model=Album,
        fields__year__group='metadata',
        fields__artist__group='metadata',
    )

    # @test
    show_output(form)
    # @end

This ends up as this documentation:

This is a normal test that runs with the normal test suite, with some additional markup:

• The triple quoted strings that are declared with # language=rst are included in the docs.
• Code is by default included in the documentation
• You can exclude code with # @test/# @end for checks you don’t want to include in the docs
• show_output renders some HTML output into a file that is then shown inline in the finished docs
• The .. uses command is used to mark what features this test uses so the examples are automatically linked from the reference API docs

With this infrastructure in place, some fixes or features can be implemented with all the required tests written as the documentation with no additional tests. This radically incentivises writing docs compared to duplicating work across tests and docs.

class WideLabelBoundField(BoundField):
    def label_tag(self, contents=None, attrs=None, label_suffix=None):
        if attrs is None:
            attrs = {}
        attrs['class'] = 'wide'
        return super().label_tag(contents, attrs, label_suffix)


class NebulaForm(Form):
    name = CharField(
        bound_field_class=WideLabelBoundField,
    )

To set a single CSS class on a single label, you have to create an entire class. Let’s look at the same thing in iommi:

class NebulaForm(Form):
    name = Field.text(
        label__attrs__class__wide=True,
    )

But, you might object, what if you need to run some code to customize it? Like if the example didn’t just set "wide" as the value, but set it to "wide" only for staff?

Not only is this also easy in iommi, I would argue it’s even easier and cleaner than in the BoundField case above:

class NebulaForm(Form):
    name = Field.text(
        label__attrs__class__wide=lambda user, **_: user.is_staff,
    )

Another very useful case for this pattern is to have a link on one page of your product with a link to a create form with prefilled data based on the context of the page you linked from. Like having an artist page with a link to the create album page with the artist filled in.

The Django admin does this, but Django forms do not. Because Django forms have an API that takes a dict for the data and not the request object itself, it can’t be retrofitted to have this feature either. It’s a nice example of where limiting the API surface area also limits future development.

In iommi, defaults-from-GET is the default for all forms. So if you build with iommi you get this feature across your product for free, not just in the admin. We even handle the edge cases for you like when you have a GET form, and you supply parameters via GET, so the form needs to know if this is from some URL or from a submit of the form. This is handled in iommi for you transparently.

This is the middleware I came up with to do this:

def domain_strip_middleware(get_response):

    def domain_strip_middleware_inner(request):
        if not settings.DEBUG:
            return get_response(request)

        m = re.match(
            r'/https?://(?P<domain>[^/]*)(?P<path>/.*)', 
            request.get_full_path()
        )
        if m:
            return HttpResponseRedirect(m.groupdict()['path'])

        return get_response(request)

    return domain_strip_middleware_inner

When I joined TriOptima back in 2010, a common pattern emerged where names of things were slightly off because of stray whitespaces. Sometimes we had duplicates like "foo", "foo " and " foo" in the database. Sometimes we couldn’t find stuff in logs because you searched for "foo was deleted", when actually you had to search for "foo was deleted" (notice the double space!). Sorting was “broken” because " foo" and "foobar" are not next to each other. And more issues that I can’t remember…

It was everywhere, causing problems across the entire code base. Each individual issue was easily fixed by cleaning up the data, but it added up to an annoying support burden. My fix at the time was to make a function that took a Django Form instance and returned a new instance with space stripping on all fields. Something like:

form = auto_strip(Form(...))

After I added that to every single Django form in the entire code base that slow and steady trickle of bugs and annoyances just stopped. From seeing a few a month consistently to zero for the next ~9 years. Even better: I never got a complaint about it.

This was fixed in Django 1.9 after some fierce debating back and forth (“will never happen” was uttered). In iommi forms we’ve had this since the beginning, which turns out to be a few months ahead of when Django took this decision (although it was tri.form and not iommi back then).

Just when you think you’re out

Unfortunately the story doesn’t end here. I started getting this issue again, and it took me a while to realize it’s because of SPA-like pages that uses Pydantic serializers instead of a form library. To solve this I added this base class for my schemas:

class Schema(ninja.Schema):
    @validator('*')
    def strip_spaces(cls, v: Any) -> Any:
        if isinstance(v, str) and '\n' not in v:
            return v.strip(' ')
        return v

The reason I’m not stripping spaces if the text contains a newline is to avoid situations where you have multiline text fields with indented code. Maybe it’s just programmers who will care, but we tend to care a lot :P

Modifying data silently and by default like this sounds like a bad idea and I also get a little pit in my stomach when I think about it with that frame of mind, but this specific case seems like all win and no downside from my 14 years of experience doing it.

• As you’re sending pixels instead of text, screen readers don’t work
• Screenshots adjust badly to zoom levels
• Responsive layouts don’t work
• Automatic dark mode selection doesn’t work
• They are larger to download
• They are hard to generate
• They are slow to generate

When writing the iommi documentation I realized that I can bypass all that by using embedded iframes instead of screenshots. Instead of spinning up a headless Chrome, writing playwright/selenium automation and suffering through all that, I can render the page I am documenting like normal, and save the html to disk, which is then linked to with an iframe. It required some custom tooling, but check out the iommi cookbook for examples and I think you’ll agree the results are pretty great.

Here’s a sample test from the cookbook that generates the iframe, and then the rST file for the docs:

def test_how_do_you_turn_off_pagination(small_discography):
    # language=rst
    """
    How do you turn off pagination?
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Specify `page_size=None`:
    """

    table = Table(
        auto__model=Album,
        page_size=None,
    )

    # @test
    show_output(table)
    # @end
    

The show_output command handles all the saving to the right filename. Our custom tooling then weaves that together in the rST output to produce the iframe.

We write all our documentation as tests first, and sometimes the documentation is all the tests we need for new features. But the details of that is the topic for another blog post 😜

I made the switch to one big file to see how I liked it, and I am very happy I did. It’s much nicer, and my work project isn’t super large, so it’s only ~630 lines anyway. It’s much nicer to find stuff and add new path mappings. Highly recommend.

Nesting

After doing this a while I noticed that this pattern came up frequently:

path('projects/<project_pk>/', ProjectPage().as_view()),
path('projects/<project_pk>/duplicate/', duplicate_project),
path('projects/<project_pk>/ev/', ev_index),
path('projects/<project_pk>/ev/edit/', ev_edit),
...

Işık reminded me that include() in Django can clean that up:

path('projects/<project_pk>/', include([
    path('', ProjectPage().as_view()),
    path('duplicate/', duplicate_project),
    path('ev/', ev_index),
    path('ev/edit/', ev_edit),
])),
...

Not super clean, but better. I also didn’t like the verboseness of the iommi views like ProjectPage().as_view() which also felt a bit messy, so after a while I wrote this:

def path(path, view_or_list, kwargs=None):
    if isinstance(view_or_list, list):
        assert kwargs is None
        return orig_path(path, include(view_or_list))
    elif isinstance(view_or_list, type):
        return orig_path(path, view_or_list().as_view(), kwargs=kwargs)
    else:
        try:
            return orig_path(path, view_or_list.as_view(), kwargs=kwargs)
        except AttributeError:
            return orig_path(path, view_or_list, kwargs=kwargs)

Now the same path mapping can look like this:

path('projects/<project_pk>/', [
    path('', ProjectPage),
    path('duplicate/', duplicate_project),
    path('ev/', [
        path('', ev_index),
        path('edit/', ev_edit),
    ]),
]),
...

The code above is hereby released in the public domain. Try it and see how you like it.