Pretendo

October Progress Update

Published by
Jon Jon
on November 1, 2024

Welcome back to another progress report. While it may seem like not a lot has been going on lately, we've been hard at work in private working on some exciting new updates to share! This blog post will serve as both an announcement for some upcoming features/additions, as well as a sneak-peak behind the scenes to see what sort of experiments we've been running!

Archiverse

As many of you know, the original Miiverse service shut down late 2017. However before the service went dark for good, a team of dedicated archivists known as Archive Team managed to archive 17TB worth of posts, drawings, and images. This data was then published onto the Internet Archive.

A group known as Archiverse was later established in 2018, providing the archived data both for modern audiences who may not have experienced Miiverse in its prime, and those feeling nostalgic who want to see themselves and what they were doing all those years ago, formally accessible at archiverse.guide. In early 2024 the original maintainer left the project, leaving it in the hands of a new team now accessible at archiverse.app.

We are happy to announce that we have come together with the new maintainers of Archiverse to merge our services! Archiverse is now an official Pretendo Network service, with the intent to have full (read-only) integrations within the on-console Miiverse apps and in games. This migration will take some time to complete, as we work towards moving many terabytes of data to our systems and finalizing the modifications needed to integrate everything, so stay tuned for a future blog post announcing the migration's completion!

Archiverse can now be visited at archiverse.pretendo.network!

Note: Archiverse data will be read-only, and will not be able to be modified by any account system, PNID or otherwise. Functions like "Yeahs", comments, etc. will be disabled when using Archiverse. The data used by Archiverse was taken from the browser version of Miiverse, meaning it lacks some critical in-game data needed for some games which may be impossible to recover, as it was only accessible via the Miiverse in-game API. Archiverse support in games will vary on a game-by-game basis depending on what data can and cannot be recovered/reconstructed.

New Members

With the aforementioned project merge with Archiverse, we'd like to welcome our two newest team members joining us along for the ride! Please welcome SuperFX and Luna, the lead maintainer and designer of Archiverse respectively. Originally only helping out with the migration, we've now offered them official spots on our development team. Both are incredibly talented web developers/designers, so expect to see some exciting new features and improvements rolling out in the future.

New Licensing

We encourage others to use our work in other 3rd party projects, such as GoCentral, custom "Rock Central" servers for Rock Band 3 on the Wii, PlayStation 3, and Xbox 360. To maintain openness, we typically license our work in ways that prevent privatization. Most of our work has been licensed under AGPLv3, as it ensures "network use" (running software on a server) is treated as "distribution", which aligns with our focus on server-based projects.

However, AGPLv3's strict requirements can be too restrictive, as it forces works which use the AGPLv3 licensed work (even as a library, or a non-derivative work) to also adopt AGPLv3 in its entirety, limiting compatibility with other, more permissive, licenses such as Apache 2.0. While this infectious nature has benefits, we recognize the need for more flexibility in certain cases. Thus, we will be reviewing some repositories when appropriate to see if less restrictive licenses could work better for specific projects.

One project, our error-codes library for Wii U/3DS, has already been re-licensed under LGPLv3, which offers similar protections to AGPLv3 but is less restrictive and avoids the full infection of AGPLv3. We will still use AGPLv3 for most projects however as other licenses lack the "network use is distribution" clause which makes it unsuitable for purely server-side software.

Enhanced Multiplayer with Relays

We have begun some experiments of a new method of handling multiplayer sessions known as "relays". The networking library Nintendo used for Nintendo Network has support for relays, however they go unused officially and many games have the relay functionality stripped out to varying degrees. We have begun our own attempt at manually recreating the concept in order to boost security and stability in the future.

Note: Relays are highly experimental, and not widely tested. We have seen promising results during some initial private tests, however these tests were tiny in scale and nothing suggests these are ready for any sort of public use. The experiment MAY be scrapped if found to be inviable in the future. Relays are not deployed on any game server, beta or otherwise, at this time.

Some background: All games on Nintendo Network which feature multiplayer are P2P (peer-to-peer). Nintendo uses very little in terms of dedicated servers, opting to only use dedicated servers for features such as object storage and leaderboards. This means that rather than your console talking to a server the entire time, your console instead talks directly to other consoles in a mesh. Our servers act as the middlemen, managing individual client connections and grouping them together in what's known as a "gathering", also called a "session". Once a gathering is created on the server, users may join it. The server then tells all the relevant clients in the session who each of the other players are (either directly on join or through live notifications after already joining), and then your consoles begin talking to each other after establishing connections following NAT traversal.

This architecture has several pros, including:

  1. Less strain on the servers
  2. Potentially lower latency for clients geographically closer to each other
  3. P2P networks tend to be very good at self-adjusting to account for dropped/new connections

However it does come with some notable cons, such as:

  1. Connection issues due to incompatible networks, resulting in failed NAT traversal (very common cause of 118-XXXX errors)
  2. Potential security concerns over direct visibility of other clients IP addresses
  3. Lack of server involvement makes cheating/hacking much easier
  4. Potentially higher latency for clients geographically farther from each other or between those with slower home internet speeds

Relays are designed to try and combat some of these cons. Rather than the matchmaking server telling each client the addresses of other players, the server would instead provide the address for a running relay server. Clients then connect to these relay servers as if they were other clients in the mesh. The server then accepts all incoming P2P packets and routes them to the other players also connected to the relay as needed. These relay servers effectively turn P2P games into client->server games.

Routing this session traffic through our own relay servers could fix many of the cons associated with P2P networking, including:

  1. Removing the issue of incompatible networks, as now you are only talking to our server and not directly to another user
  2. Effectively masking your IP address from other users, as now the only addresses being used would be ours
  3. Logging of gameplay events to look for anomalies, and even logging entire sessions for later review in the style of Hypixel Atlas "Replays" to more easily track and report cheaters
  4. More efficient player kicks on a per-session basis

Early tests in Splatoon showed promising results, being able to complete several test matches while running through relays with no direct connections between users. However this system is still in it's infancy and many technical and design quirks have not been ironed out, including managing many relays at scale, increased cost estimations due to increased data usage on our servers, etc. We do not have an ETA for when this system will arrive or in what shape/form it would take.

New Games and Updated Matchmaking

At the moment we are prioritizing improving our internal tooling, underlying networking libraries, game server protocol implementations, and more rather than prioritizing the total number of games we output. However we may decide to release new games if it relates to those efforts, such as games which use features which have yet to be tested, or simple games used to test large-scale changes just as reworks to common protocols.

As of today we are releasing 2 new games into beta testing:

  • Monster Hunter 4 Ultimate
  • Super Street Fighter IV

These games had their servers created using some experimental internal tooling we are developing to help streamline to creation of simple game servers. In these cases, the backends for these servers were entirely generated automatically! These servers also serve as a test for the large-scale matchmaking rework that DaniElectra has been working on the past few months!

These matchmaking reworks bring not only better matchmaking logic in general, but also more stable session tracking and creation. These changes give us more control over match management and can be used for more accurate player tracking/moderation, even in cases where players may try to anonymize themselves or mask their user information in local connections.

Pending the results of the matchmaking tests, existing games may soon be getting the same matchmaking rework to try and improve the quality of matchmaking. Similarly, smaller/simple games may be released more often pending the results of our new internal tooling.

Additionally, Minecraft: Wii U Edition has been upgraded with this matchmaking rework, fixing the bug where only 2 players can connect at a time. These changes are available to the public for more widespread testing.

Finally, we have recently documented all the official names for 118-XXXX error codes. Using this knowledge, we have begun to identify some potential causes for several of these errors. These are now also being looked into, in an effort to improve our matchmaking logic and reduce the number of incompatible clients being put in the same sessions. For details, see the official feature enhancement request on GitHub.

Conclusion

While this month did not see much in the way of new public features, we've been hard at work on many internal affairs. We hope you enjoyed our first more "technical"/"behind the scenes" blog post, and we hope to add more updates like these to future blog posts, mixed in with actual content updates!