Feature Proposal: Q-Factor Parsing In Accept-Encoding Header

Hey everyone! Today, we're diving into a feature proposal that aims to enhance how our ZstdMiddleware handles content encoding. This is all about giving clients more control over their preferred compression methods. Let's get into the details!

Introduction

First off, massive thanks for providing such a stellar and high-performance middleware! Seriously, it's been a game-changer. Now, let's talk about a potential improvement. Currently, the ZstdMiddleware smartly picks a compression method by quickly checking if "zstd" or "gzip" are present in the Accept-Encoding header. This server-preferred approach is super speedy, which is fantastic. However, it kinda overlooks the client's actual preferences, especially those defined by the quality value (q-factor) as outlined in RFC 9110 (Section 12.5.3. Accept-Encoding).

Understanding the Issue

The current method is like offering a menu where you only look at the dish names and ignore the customer's preference ranking. Imagine a client sends this: Accept-Encoding: gzip;q=1.0, br;q=0.9, zstd;q=0.1. This client is shouting from the rooftops that they really prefer gzip over zstd – a whole 10 times more! But because "zstd" is in the mix, our middleware might jump to using Zstd, which isn't ideal. We want to be more client-friendly, right?

The Proposal: Respecting Q-Factors

So, here’s the meat of the proposal: I'm suggesting we add a new, optional boolean flag to the ZstdMiddleware constructor. Think of it as a switch that lets us choose between speed and client preference. We could call it respect_q_factors: bool = False. The beauty of this is its flexibility. When respect_q_factors=False (which would be the default), the middleware will keep doing its lightning-fast check, just like it does now. This means zero performance hiccups for existing users – win-win!

Diving Deeper into respect_q_factors=True

Now, when respect_q_factors=True (this is the opt-in part), the middleware will put on its thinking cap and use a slightly more sophisticated (but still efficient) parsing logic. It'll properly read those q-factors and then pick the encoding (zstd or gzip) that boasts the highest q-value among the encodings our server supports. It’s like having a smart waiter who knows exactly what you want!

The Code Example: Making it Real

To illustrate, here’s a snippet of how this might look in Python:

class ZstdMiddleware:
    def __init__(
        self,
        app: ASGIApp,
        level: int = 3,
        # ...
        gzip_fallback: bool = True,
        respect_q_factors: bool = False,  # <-- New flag
        excluded_handlers: Union[List, None] = None,
    ) -> None:
        # ...
        self.gzip_fallback = gzip_fallback
        self.respect_q_factors = respect_q_factors
        # ...

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        if self._is_handler_excluded(scope) or scope["type"] != "http":
            return await self.app(scope, receive, send)

        accept_encoding = Headers(scope=scope).get("Accept-Encoding", "")

        if self.respect_q_factors:
            # Slower path: Parse q-factors to find the best match
            preferred_encoding = self._get_preferred_encoding(accept_encoding)
            
            if preferred_encoding == "zstd":
                responder = ZstdResponder(self.app, ...)
                await responder(scope, receive, send)
                return
            if self.gzip_fallback and preferred_encoding == "gzip":
                responder = GZipResponder(self.app, self.minimum_size)
                await responder(scope, receive, send)
                return
        
        else:
            # Fast path: Current behavior (default)
            if "zstd" in accept_encoding:
                responder = ZstdResponder(self.app, ...)
                await responder(scope, receive, send)
                return
            if self.gzip_fallback and "gzip" in accept_encoding:
                responder = GZipResponder(self.app, self.minimum_size)
                await responder(scope, receive, send)
                return

        await self.app(scope, receive, send)
    
    def _get_preferred_encoding(self, accept_encoding: str) -> str:
        # A helper function to parse the header and return the
        # highest-q-value encoding supported ("zstd", "gzip", or "")
        # ...
        pass

Let's break this down a bit. We're adding a respect_q_factors flag to the ZstdMiddleware initialization. This flag determines the path the middleware takes when processing requests. If respect_q_factors is set to True, the middleware will invoke the _get_preferred_encoding method to parse the Accept-Encoding header and determine the most preferred encoding based on the q-factors. This ensures that the server respects the client's preferences for content encoding. Conversely, if respect_q_factors is False (the default), the middleware will continue to use the existing fast-path logic, checking for the presence of `