Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] TypeError: can not serialize 'UUID' object models with pk type uuid #154

Open
pollitux opened this issue Sep 11, 2022 · 2 comments · May be fixed by #156
Open

[BUG] TypeError: can not serialize 'UUID' object models with pk type uuid #154

pollitux opened this issue Sep 11, 2022 · 2 comments · May be fixed by #156
Labels
Needs investigation

Comments

@pollitux
Copy link

Describe the bug
When looking at a model and it has a pk type UUID it causes the exception TypeError: can not serialize 'UUID' object in chanels redis

class MyModel(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)

LOG

File "/code/djangochannelsrestframework/observer/model_observer.py", line 143, in post_change_receiver
mmcb          |     self.send_messages(
mmcb          |   File "/code/djangochannelsrestframework/observer/model_observer.py", line 166, in send_messages
mmcb          |     async_to_sync(channel_layer.group_send)(group_name, message_to_send)
mmcb          |   File "/usr/local/lib/python3.10/site-packages/asgiref/sync.py", line 218, in __call__
mmcb          |     return call_result.result()
mmcb          |   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
mmcb          |     return self.__get_result()
mmcb          |   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
mmcb          |     raise self._exception
mmcb          |   File "/usr/local/lib/python3.10/site-packages/asgiref/sync.py", line 284, in main_wrap
mmcb          |     result = await self.awaitable(*args, **kwargs)
mmcb          |   File "/code/channels_redis/core.py", line 682, in group_send
mmcb          |     ) = self._map_channel_keys_to_connection(channel_names, message)
mmcb          |   File "/code/channels_redis/core.py", line 782, in _map_channel_keys_to_connection
mmcb          |     channel_key_to_message[key] = self.serialize(value)
mmcb          |   File "/code/channels_redis/core.py", line 802, in serialize
mmcb          |     value = msgpack.packb(message, use_bin_type=True)
mmcb          |   File "/usr/local/lib/python3.10/site-packages/msgpack/__init__.py", line 38, in packb
mmcb          |     return Packer(**kwargs).pack(o)
mmcb          |   File "msgpack/_packer.pyx", line 294, in msgpack._cmsgpack.Packer.pack
mmcb          |   File "msgpack/_packer.pyx", line 300, in msgpack._cmsgpack.Packer.pack
mmcb          |   File "msgpack/_packer.pyx", line 297, in msgpack._cmsgpack.Packer.pack
mmcb          |   File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
mmcb          |   File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
mmcb          |   File "msgpack/_packer.pyx", line 291, in msgpack._cmsgpack.Packer._pack
mmcb          | TypeError: can not serialize 'UUID' object

Additional context
This is my code that I use to observe the model with pk type UUID

from djangochannelsrestframework.generics import GenericAsyncAPIConsumer
from djangochannelsrestframework.observer.generics import ObserverModelInstanceMixin

from .models import MyModel
from .serializers import MySerializer


class MyRetrieveConsumer(ObserverModelInstanceMixin, GenericAsyncAPIConsumer):
    queryset = MyModel.objects.all()
    serializer_class = MySerializer

I solved it by checking if the pk value of the model is a UUID type and convert it to a string

class ModelObserver(BaseObserver):
    .
    .
    .
    .
    .
    .

    def serialize(self, instance, action, **kwargs) -> Dict[str, Any]:
        message_body = {}
        if self._serializer:
            message_body = self._serializer(self, instance, action, **kwargs)
        elif self._serializer_class:
            message_body = self._serializer_class(instance).data
        else:
            message_body["pk"] = instance.pk

        # THIS IS MY SOLUTION FOR NOW
        # Check if the pk of the model is UUID type and convert it to a string
        if isinstance(message_body['pk'], UUID):
            message_body.update({'pk': str(message_body['pk'])})

        message = dict(
            type=self.func.__name__.replace("_", "."),
            body=message_body,
            action=action.value,
        )

        return message
pollitux added a commit to pollitux/djangochannelsrestframework that referenced this issue Sep 11, 2022
@pollitux
Check if the pk of the model is type uuid and convert it to a str closes
64cf570 
 NilCoalescing#154
@pollitux pollitux mentioned this issue Sep 11, 2022
Closed
pollitux added a commit to pollitux/djangochannelsrestframework that referenced this issue Sep 11, 2022
@pollitux
Check if the pk of the model is type uuid and convert it to a str closes
c8d4d3b 
 NilCoalescing#154
@pollitux pollitux linked a pull request Sep 11, 2022 that will close this issue
Open
@befocken
Copy link

I think I actually have a similar issue, but with a different field type.

In my case, I set the COERCE_DECIMAL_TO_STRING of the REST_FRAMEWORK settings to FALSE, as we would like to use actual numbers in JSON format, not a stringified version.
If I use the view_as_consumer function to wrap the underlying ViewSet of the model we created for the REST API, it does not raise the same error, but instead it just wraps the whole data object in quotes... This then looks like the following:

{
  "errors": [],
  "data": "{\"test\": \"test\"}",
  "action": "list",
  "response_status": 200,
  "request_id": "12345"
}

This is of course also not the expected behavior...

That's why I investigated a little and tried setting COERCE_DECIMAL_TO_STRING to TRUE again (the default value). Then, the JSON is correctly returned.

As I tested a little more with an explicit Consumer with the queryset and the serializer_class set in the same way as in the ViewSet used in the view_as_consumer call, a similar exception to the one mentioned in this issue is thrown, but instead of UUID object, it remarks Object of type Decimal is not JSON serializable.

This might therefore need another test and special handling like the one introduced in #155, correct?

@khamaileon
Copy link

@pollitux If I understand your problem well, it seems to me that it comes from the serialization. msgpack works only with primitive types. You can try with another serialization library. Personally I use orjson.

class MyModelConsumer(ObserverModelInstanceMixin, GenericAsyncAPIConsumer):
    queryset = MyModel.objects.all()
    serializer_class = MyModelSerializer

    @classmethod
    async def decode_json(cls, text_data):
        return orjson.loads(text_data)

    @classmethod
    async def encode_json(cls, content):
        return orjson.dumps(content).decode()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Needs investigation
Projects
None yet
Milestone
No milestone
3 participants
@khamaileon @pollitux @befocken