AttributeError: 'NoneType' object has no attribute 'uri_domain' (in call to /api/v1/timelines/home)

[virtualmarc]

When loading the Home Timeline Takahe 0.10.1 responds with an Error 500 and the following Traceback:

ERROR:django.request:Internal Server Error: /api/v1/timelines/home
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/api/decorators.py", line 46, in inner
    return function(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/hatchway/view.py", line 302, in __call__
    response = self.view(request, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/api/views/timelines.py", line 46, in home
    schemas.Status.map_from_timeline_event(pager.results, request.identity),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/api/schemas.py", line 244, in map_from_timeline_event
    return [
           ^
  File "/takahe/api/schemas.py", line 245, in <listcomp>
    cls.from_timeline_event(
  File "/takahe/api/schemas.py", line 227, in from_timeline_event
    **timeline_event.to_mastodon_status_json(
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/activities/models/timeline_event.py", line 260, in to_mastodon_status_json
    return self.subject_post_interaction.to_mastodon_status_json(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/activities/models/post_interaction.py", line 560, in to_mastodon_status_json
    post_json = self.post.to_mastodon_json(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/activities/models/post.py", line 1166, in to_mastodon_json
    "content": self.safe_content_remote(),
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/activities/models/post.py", line 453, in safe_content_remote
    return self.safe_content(local=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/activities/models/post.py", line 440, in safe_content
    return func(local=local)
           ^^^^^^^^^^^^^^^^^
  File "/takahe/activities/models/post.py", line 408, in _safe_content_note
    return ContentRenderer(local=local).render_post(self.content, self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/takahe/core/html.py", line 343, in render_post
    uri_domain=(None if self.local else post.author.domain.uri_domain),
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'uri_domain'

This suddenly startet around 12h ago for one user, so I think an incomplete post came in at that time.
I had no problems loading the local or federated timeline. Other user's had no problems loading their timelines so far.

Now, 12h later, the affected user is able to load his home timeline again, so I think this post is now out of the view.
If I scroll down far enough to the posts around 12h before, with the affected user, the error occurs again.

I tried to find the Post in the Django Admin console with no luck, but I guessed that this is an identity without a domain, so I selected users_identity with domain_id is null and got 574 rows.

[alphatownsman]

@virtualmarc 574 is a lot. wondering if it has been a bug.

you may select actor_uri of these identities see if there's a pattern; you may also call identity.fetch_actor() for each of those identities see if that will fix things by itself.

[virtualmarc]

@alphatownsman I just checked some of them (not all since these are too many).

Some are using software that is currently incompatible with Takahe and are probably fixed with #682 (not only GoToSocial, but also bird.makeup, a.gup.pe, some other twitter mirrors. some are friendica or write.as, where I haven't actively observed federation issues but am not sure if they really work).

Most of them are instances not available anymore (either responding with 404, timing out or no dns records anymore).

One I checked was a moved profile on Mastodon.

Some few are normal Mastodon accounts that are still available.

While looking at the Activity Pub JSON's of some of the identities, they sometimes look weird.

This one for example is from a friendica instance missing most information:

{'id': 'https://squeet.me/profile/tagesschau', 'type': 'Person', 'inbox': None, 'outbox': None, 'featured': None, 'followers': None, 'following': None, 'preferredUsername': None, 'publicKey': {'id': None, 'owner': 'https://squeet.me/profile/tagesschau', 'publicKeyPem': None}, 'published': '2023-05-09T12:05:16Z', 'url': None, 'toot:discoverable': True}

This one is from a mastodon instance from a moved account, also missing most information:

{'id': 'https://nafo.uk/users/Burito_Bandito_', 'type': 'Person', 'inbox': None, 'outbox': None, 'featured': None, 'followers': None, 'following': None, 'preferredUsername': None, 'publicKey': {'id': None, 'owner': 'https://nafo.uk/users/Burito_Bandito_', 'publicKeyPem': None}, 'published': '2023-12-10T14:48:06Z', 'url': None, 'toot:discoverable': True}

When fetching some of the actor uri's of failing mastodon instances with curl (like the nafo.uk example or at cupoftea.social) I'm getting the browser check page of Cloudflare, this can also be a problem that the request is blocked by the Cloudflare (or any other CDN) WAF (at least when I try it from my server, fetching fine from my home connection, so the WAF is probably blocking the datacenter ip's).

[alphatownsman]

Thanks.

at least one of these domain are fixable, which is bird.makeup who only accepts application/activity+json, not application/ld+json.

for the rest, Fediverse data are always chaos, guess we just need to use actor_url_parts.hostname as fallback when we have to.

I'll put some patches together.