TODO: IDEA: To keep other servers from not re-requesting the idcert after the ttls has passed, the
idcert should have some sort of timestamp that is signed by the original server, so that clients can
verify that a server has the most up-to-date idcert cached for a user -flori
-
Info
A failed signature verification does not always mean that the message is invalid. It may be that
@@ -3344,16 +3339,179 @@
6.3 Private key
Delete encrypted private key material
Get encrypted private key material upload size limit
-
6.4 Cryptographic recommendations
+
6.4 Caching of ID-Certs
+
The caching of ID-Certs is an important mechanism in polyproto to aid in fairly distributing the load
+generated by ID-Cert lookups to the servers generating the traffic, not to the server the ID-Cert
+is actually from. This practice should help make the operation of low-resource home servers, used
+exclusively for hosting identities, more viable.
+
This section of the protocol definition defines required behaviors related to the correct caching
+of ID-Certs for both home servers and clients.
+
To make this section more understandable, we will bring back the example from section 6.2.1:
+
+Revisiting the example scenario from section 6.2.1
+
+
Example
+
Say we have two actors. Alice, who is registered on Server A, and Bob, who is registered
+on Server B. Alice and Bob are having a conversation on Server B. Given a signed message
+from Alice, such as Bob would receive from Server B, the process of verifying the signature
+would look like this:
+
sequenceDiagram
+autonumber
+
+actor b as Bob
+participant sb as Server B
+participant sa as Server A
+
+sb->>b: Alice's signed message
+opt Server A's ID-Cert is not cached on Bob's client
+ b->>sa: Request Server A ID-Cert
+ sa->>b: Server A ID-Cert
+end
+opt Alice's ID-Cert is not cached on Bob's client
+ b->>sb: Request Alice's ID-Cert
+ opt Alice's ID-Cert is not cached on Server B
+ sb->>sa: Request Alice's ID-Cert
+ sa->>sb: Alice's ID-Cert
+ end
+ sb->>b: Alice's ID-Cert
+end
+b->>b: Verify signature of Alice's message (Fig. 4)
+
Fig. 3: Sequence diagram of a successful message signature verification.
+
+
In the case where alice@server-a.example.com and bob@server-b.example.com are having a
+conversation where the communications server is any server other than server-a.example.com,
+Bob should request Alice's ID-Cert from that server first, instead of from server-a.example.com.
+
+Further notes on why we consider this cached distribution process a good idea
+Bob's client could request Alice's public identity key from Server A, instead of Server B.
+However, this is discouraged, as it
+
+- Generates unnecessary load on Server A; Doing it this way distributes the load of public
+ identity key requests more fairly, as the server that the message was sent on is the one that
+ has to process the bulk of public identity certificate requests.
+- Would expose unnecessary metadata to Server A; Server A does not need to know who exactly
+ Alice is talking to, and when. Only Server B, Alice and Bob need to know this information.
+ Always requesting the public identity key from Server A might expose this information to
+ Server A.
+
+Clients should only use Server A as a fallback for public identity key verification, if Server B
+does not respond to the request for Alice's public identity key, or if the verification fails
+with the public identity key from Server B. Security considerations listed in this section of
+the protocol definition ensure that this cached distribution process is safe and trustworthy
+
+
Both Bob's client and Server B should now cache Server A's and Alice's ID-Certs, to avoid having to
+request them again.
+
The TTL (time to live) of these cached items should be relatively short. Recommended values
+are between one (1) and twelve (12) hours. Cached ID-Certs must be evicted from
+the cache, after the TTL has expired. Expired cached ID-Certs must not be used for signature
+verification of new messages, even if the client cannot renew its cache. All of this applies to both
+servers and clients. The TTL for a certificates' cache duration is dictated by the home server, which
+that certificate has been issued by. You can read more on that in
+subsection 1 of this section.
+
+Why not select longer lived TTLs for cached ID-Certs?
+Suppose that an actors' private identity key is compromised. The actor notices this, and revokes
+their ID-Cert. If the TTL of cached ID-Certs is too long, the compromised ID-Cert might still be
+used for signature verification for a long amount of time, even after the ID-Cert has been revoked.
+This is a problem in the following hypothetical scenario with malicious actor "Eve" and victim
+"Alice":
+
+Downside of using higher values for a TTL
+
+- One of Alice's private identity keys is compromised.
+- Malicious actor Eve logs onto Server X, which Alice has never connected to before, using
+ Alice's ID-Cert of which the corresponding private identity key has been compromised.
+- In the meantime, Alice notices the breach, requesting the revocation of her ID-Cert on
+ all servers she is connected to.
+- Server X does not get this revocation message, as Alice does not know about her connection
+ to Server X, where Eve is impersonating Alice.
+- Eve can now impersonate Alice on Server X for as long as the TTL of the cached ID-Cert on
+ Server X has not expired. With a high value, this could be a long time.
+
+
+
+
If the verification fails, Bob's client should try to re-request the key from Server B first.
+Should the verification fail again, Bob's client can try to request Alice's public identity key
+and ID-Cert from Server A (Alice's home server). The signature verification process should then be
+re-tried. Should the verification still not succeed, the message should be treated with extreme
+caution.
+
sequenceDiagram
+autonumber
+
+actor b as Bob
+participant sb as Server B
+participant sa as Server A
+
+b->>b: Verify signature of Alice's message, attempt 1
+alt Verification fails
+ b->>sb: Request Alice's ID-Cert
+ sb->>b: Alice's ID-Cert
+ b->>b: Verify signature of Alice's message, attempt 2
+ opt Verification fails again
+ b->>sa: Request Alice's ID-Cert
+ sa->>b: Alice's ID-Cert
+ b->>b: Verify signature of Alice's message, final attempt
+ opt Verification is still unsuccessful
+ b-->b: Treat Alice's message with extreme caution.
+ end
+ end
+else Verification succeeds
+ b-->b: Treat Alice's message as verified.
+end
+
Fig. 4: Sequence diagram showing how message verification should be handled if the first attempt
+to verify the signature fails.
+
After evicting a cached ID-Cert:
+
+- A client should request an up-to-date ID-Cert of the target actor from the server where the actor
+ was last seen by the client.
+- A server should request an up-to-date ID-Cert from the target actors' home server.
+
+
+
Info
+
It is not of vital importance that a client requests an ID-Cert of an actor whose ID-Cert has
+just been evicted from the cache from the server, where the actor was last seen by the client
+precisely. This means, that a client application doesn't necessarily need to update an internal
+state of where that actor has last been seen every single time that actor sends a message somewhere.
+This internal state update could instead happen every 5, 30, or even 60 seconds. What is
+important, however, is that this state update does eventually happen within a reasonable amount
+of time, to help achieve the goal of dynamic server load distribution.
+
+
6.4.1 Verifying that a newly retrieved ID-Cert is not out-of-date
+
While the goal of achieving dynamic server load distribution to increase the viability of small,
+low-resource home servers is a noble one, this goal must not undermine P2s trust model, which other
+aspects of the protocol work very hard to uphold. Retrieving ID-Certs from a middleman introduces
+a new attack surface which must be mitigated. Consider the following example:
+
+Example attack abusing blind middleman trust
+
+- One of Alice's private identity keys is compromised.
+- Malicious actor Eve logs onto a malicious Server X which is controlled by Eve, impersonating
+ Alice by using Alice's ID-Cert of which the corresponding private identity key has been compromised.
+- In the meantime, Alice notices the breach, requesting the revocation of her ID-Cert on
+ all servers she is connected to.
+- Server X does not care for this revocation message, as it is malicious (attacker controlled)
+- Eventually, the TTL for this compromised certificate expires. Users on Server X contact the
+ server for the latest certificate of Alice.
+- Server X responds with the compromised ID-Cert, claiming that this is the most up-to-date
+ ID-Cert, even though it has been revoked.
+- Through all users trusting Server X blindly, Eve and Server X can impersonate Alice for as
+ long as Alice's compromised ID-Cert would have been valid for (valid-not-after attribute in X.509
+ certificates). Until then, users do not notice that this certificate has been revoked and
+ should no longer be valid.
+
+
+
This kind of attack has been considered and mitigated in polyproto. bla
+
6.5 Cryptographic recommendations
For two implementations of polyproto to be interoperable, they must support an overlapping set of
digital signature algorithms.
If technically practical, all implementations of polyproto must support use of the Ed25519 digital
signature algorithm for signing messages and generating ID-Certs. The use of the RSA algorithm for
digital signatures is heavily discouraged.
-
6.5 Best practices
+
6.6 Best practices
The following subsections are dedicated to documenting best practices to consider when
implementing polyproto.
-
6.5.1 Signing keys and ID-Certs
+
6.6.1 Signing keys and ID-Certs
- When a server is asked to generate a new ID-Cert for an actor, it must make sure that the CSR is
valid and, if set, has an expiry date less than or equal to the expiry date of the server's own ID-Cert.
@@ -3366,12 +3524,12 @@
+6.6.2 Home server operation and design
- Use a caching layer for your home server to handle the potentially large amount of requests for
ID-Certs without putting unnecessary strain on the database.
-6.5.3 Private key loss prevention and private key recovery
+6.6.3 Private key loss prevention and private key recovery
- It is a good idea for home servers to limit the upload size and available upload slots for encrypted
private identity keys.
@@ -3889,7 +4047,7 @@ 9.1.1 Changing a primary servic
- 2024-12-30
+ 2024-12-30
diff --git a/feed_rss_created.xml b/feed_rss_created.xml
index 8fc5831e..249b0249 100644
--- a/feed_rss_created.xml
+++ b/feed_rss_created.xml
@@ -1 +1 @@
- Polyphonyhttps://github.com/polyphony-chat/docsen Mon, 30 Dec 2024 14:31:24 -0000 Mon, 30 Dec 2024 14:31:24 -0000 1440 MkDocs RSS plugin - v1.17.1 None Polyphony - NLnet grant application bitfl0wer polyproto updates <h1>NLnet grant application</h1><p>The <a href="https://nlnet.nl/">NLnet foundation</a> is a non-profit organization that supports open-source projects. They have agrant program that funds projects that align with their goals. On behalf of Polyphony and thepolyproto project, I have submitted an application for a grant of 10,000€ from the NLnet foundationin their funding round of October 2024.</p> Mon, 14 Oct 2024 00:00:00 +0000
- polyproto extensions bitfl0wer polyproto <h1>polyproto extensions.</h1><p>polyproto is a new federation protocol. Its main focus is enabling seamless participation of oneactor on many different servers. The core specification lacks routes for sending any sort of usergenerated data anywhere, though. What is up with that?</p> Sat, 01 Jun 2024 00:00:00 +0000
- Work on polyproto and a "vacation" ⛱️ bitfl0wer polyproto updates <h1>Work on polyproto and taking a break</h1><p>In this little update post I write about what I've done in the last couple of weeks alongside talking about taking just a little break(don't worry, y'all are not getting rid of me!)</p> Wed, 06 Mar 2024 00:00:00 +0000
- X.509 in polyproto bitfl0wer X.509 polyproto <h1>Certificates, please: X.509 in polyproto</h1><p>This blog post covers a bit about how and why X.509 is used in polyproto, and how we try to makethe process of implementing your own server and incorporating it into an existing network a littleeasier.</p> Mon, 19 Feb 2024 00:00:00 +0000
- Account migration in polyproto bitfl0wer polyproto <p>Account migration is an important and difficult thing to get right in federated systems. In this blogpost, I will outline how I imagine account migration to work in polyproto, and what benefits thisapproach brings.</p> Wed, 07 Feb 2024 00:00:00 +0000
- Porting chorus to WebAssembly + Client Update bitfl0wer chorus polyphony updates <h1>Porting chorus to WebAssembly + Client Update</h1><p>What the current state of GUI libraries in Rust means for Polyphony and chorus, and why we are porting chorus to WebAssembly.</p> Thu, 23 Nov 2023 00:00:00 +0000
- Getting started with the Polyphony Client bitfl0wer polyphony updates <h1>Getting started with the Polyphony Client</h1> Sat, 02 Sep 2023 00:00:00 +0000
- chorus Alpha 0.1.0 bitfl0wer chorus updates <h1>chorus Alpha 0.1.0</h1><p>We are alpha now! As of 2 days ago, the first Alpha of Chorus, Version 0.1.0, has been released for everyone to look at and use on crates.io!</p> Tue, 29 Aug 2023 00:00:00 +0000
- Self-updating structs, moving blog posts to GitHub, and more! bitfl0wer chorus updates <h1>Self-updating structs, moving blog posts to GitHub, and more!</h1><p>Introducing self-updating structs, explaining how they work, and what they are good for. Also, moving blog posts to GitHub, and other improvements.</p> Thu, 17 Aug 2023 00:00:00 +0000
\ No newline at end of file
+ Polyphonyhttps://github.com/polyphony-chat/docsen Mon, 30 Dec 2024 19:12:06 -0000 Mon, 30 Dec 2024 19:12:06 -0000 1440 MkDocs RSS plugin - v1.17.1 None Polyphony - NLnet grant application bitfl0wer polyproto updates <h1>NLnet grant application</h1><p>The <a href="https://nlnet.nl/">NLnet foundation</a> is a non-profit organization that supports open-source projects. They have agrant program that funds projects that align with their goals. On behalf of Polyphony and thepolyproto project, I have submitted an application for a grant of 10,000€ from the NLnet foundationin their funding round of October 2024.</p> Mon, 14 Oct 2024 00:00:00 +0000
- polyproto extensions bitfl0wer polyproto <h1>polyproto extensions.</h1><p>polyproto is a new federation protocol. Its main focus is enabling seamless participation of oneactor on many different servers. The core specification lacks routes for sending any sort of usergenerated data anywhere, though. What is up with that?</p> Sat, 01 Jun 2024 00:00:00 +0000
- Work on polyproto and a "vacation" ⛱️ bitfl0wer polyproto updates <h1>Work on polyproto and taking a break</h1><p>In this little update post I write about what I've done in the last couple of weeks alongside talking about taking just a little break(don't worry, y'all are not getting rid of me!)</p> Wed, 06 Mar 2024 00:00:00 +0000
- X.509 in polyproto bitfl0wer X.509 polyproto <h1>Certificates, please: X.509 in polyproto</h1><p>This blog post covers a bit about how and why X.509 is used in polyproto, and how we try to makethe process of implementing your own server and incorporating it into an existing network a littleeasier.</p> Mon, 19 Feb 2024 00:00:00 +0000
- Account migration in polyproto bitfl0wer polyproto <p>Account migration is an important and difficult thing to get right in federated systems. In this blogpost, I will outline how I imagine account migration to work in polyproto, and what benefits thisapproach brings.</p> Wed, 07 Feb 2024 00:00:00 +0000
- Porting chorus to WebAssembly + Client Update bitfl0wer chorus polyphony updates <h1>Porting chorus to WebAssembly + Client Update</h1><p>What the current state of GUI libraries in Rust means for Polyphony and chorus, and why we are porting chorus to WebAssembly.</p> Thu, 23 Nov 2023 00:00:00 +0000
- Getting started with the Polyphony Client bitfl0wer polyphony updates <h1>Getting started with the Polyphony Client</h1> Sat, 02 Sep 2023 00:00:00 +0000
- chorus Alpha 0.1.0 bitfl0wer chorus updates <h1>chorus Alpha 0.1.0</h1><p>We are alpha now! As of 2 days ago, the first Alpha of Chorus, Version 0.1.0, has been released for everyone to look at and use on crates.io!</p> Tue, 29 Aug 2023 00:00:00 +0000
- Self-updating structs, moving blog posts to GitHub, and more! bitfl0wer chorus updates <h1>Self-updating structs, moving blog posts to GitHub, and more!</h1><p>Introducing self-updating structs, explaining how they work, and what they are good for. Also, moving blog posts to GitHub, and other improvements.</p> Thu, 17 Aug 2023 00:00:00 +0000
\ No newline at end of file
diff --git a/feed_rss_updated.xml b/feed_rss_updated.xml
index a4345629..96beb6df 100644
--- a/feed_rss_updated.xml
+++ b/feed_rss_updated.xml
@@ -1 +1 @@
- Polyphonyhttps://github.com/polyphony-chat/docsen Mon, 30 Dec 2024 14:31:24 -0000 Mon, 30 Dec 2024 14:31:24 -0000 1440 MkDocs RSS plugin - v1.17.1 None Polyphony - NLnet grant application bitfl0wer polyproto updates <h1>NLnet grant application</h1><p>The <a href="https://nlnet.nl/">NLnet foundation</a> is a non-profit organization that supports open-source projects. They have agrant program that funds projects that align with their goals. On behalf of Polyphony and thepolyproto project, I have submitted an application for a grant of 10,000€ from the NLnet foundationin their funding round of October 2024.</p> Mon, 14 Oct 2024 15:34:49 +0000
- polyproto extensions bitfl0wer polyproto <h1>polyproto extensions.</h1><p>polyproto is a new federation protocol. Its main focus is enabling seamless participation of oneactor on many different servers. The core specification lacks routes for sending any sort of usergenerated data anywhere, though. What is up with that?</p> Sat, 01 Jun 2024 09:17:57 +0000
- Work on polyproto and a "vacation" ⛱️ bitfl0wer polyproto updates <h1>Work on polyproto and taking a break</h1><p>In this little update post I write about what I've done in the last couple of weeks alongside talking about taking just a little break(don't worry, y'all are not getting rid of me!)</p> Thu, 07 Mar 2024 21:36:03 +0000
- Self-updating structs, moving blog posts to GitHub, and more! bitfl0wer chorus updates <h1>Self-updating structs, moving blog posts to GitHub, and more!</h1><p>Introducing self-updating structs, explaining how they work, and what they are good for. Also, moving blog posts to GitHub, and other improvements.</p> Wed, 06 Mar 2024 22:42:28 +0000
- chorus Alpha 0.1.0 bitfl0wer chorus updates <h1>chorus Alpha 0.1.0</h1><p>We are alpha now! As of 2 days ago, the first Alpha of Chorus, Version 0.1.0, has been released for everyone to look at and use on crates.io!</p> Wed, 06 Mar 2024 22:42:28 +0000
- Getting started with the Polyphony Client bitfl0wer polyphony updates <h1>Getting started with the Polyphony Client</h1> Wed, 06 Mar 2024 22:42:28 +0000
- Porting chorus to WebAssembly + Client Update bitfl0wer chorus polyphony updates <h1>Porting chorus to WebAssembly + Client Update</h1><p>What the current state of GUI libraries in Rust means for Polyphony and chorus, and why we are porting chorus to WebAssembly.</p> Wed, 06 Mar 2024 22:42:28 +0000
- X.509 in polyproto bitfl0wer X.509 polyproto <h1>Certificates, please: X.509 in polyproto</h1><p>This blog post covers a bit about how and why X.509 is used in polyproto, and how we try to makethe process of implementing your own server and incorporating it into an existing network a littleeasier.</p> Tue, 20 Feb 2024 10:10:12 +0000
- Account migration in polyproto bitfl0wer polyproto <p>Account migration is an important and difficult thing to get right in federated systems. In this blogpost, I will outline how I imagine account migration to work in polyproto, and what benefits thisapproach brings.</p> Thu, 08 Feb 2024 16:39:01 +0000
\ No newline at end of file
+ Polyphonyhttps://github.com/polyphony-chat/docsen Mon, 30 Dec 2024 19:12:06 -0000 Mon, 30 Dec 2024 19:12:06 -0000 1440 MkDocs RSS plugin - v1.17.1 None Polyphony - NLnet grant application bitfl0wer polyproto updates <h1>NLnet grant application</h1><p>The <a href="https://nlnet.nl/">NLnet foundation</a> is a non-profit organization that supports open-source projects. They have agrant program that funds projects that align with their goals. On behalf of Polyphony and thepolyproto project, I have submitted an application for a grant of 10,000€ from the NLnet foundationin their funding round of October 2024.</p> Mon, 14 Oct 2024 15:34:49 +0000
- polyproto extensions bitfl0wer polyproto <h1>polyproto extensions.</h1><p>polyproto is a new federation protocol. Its main focus is enabling seamless participation of oneactor on many different servers. The core specification lacks routes for sending any sort of usergenerated data anywhere, though. What is up with that?</p> Sat, 01 Jun 2024 09:17:57 +0000
- Work on polyproto and a "vacation" ⛱️ bitfl0wer polyproto updates <h1>Work on polyproto and taking a break</h1><p>In this little update post I write about what I've done in the last couple of weeks alongside talking about taking just a little break(don't worry, y'all are not getting rid of me!)</p> Thu, 07 Mar 2024 21:36:03 +0000
- Self-updating structs, moving blog posts to GitHub, and more! bitfl0wer chorus updates <h1>Self-updating structs, moving blog posts to GitHub, and more!</h1><p>Introducing self-updating structs, explaining how they work, and what they are good for. Also, moving blog posts to GitHub, and other improvements.</p> Wed, 06 Mar 2024 22:42:28 +0000
- chorus Alpha 0.1.0 bitfl0wer chorus updates <h1>chorus Alpha 0.1.0</h1><p>We are alpha now! As of 2 days ago, the first Alpha of Chorus, Version 0.1.0, has been released for everyone to look at and use on crates.io!</p> Wed, 06 Mar 2024 22:42:28 +0000
- Getting started with the Polyphony Client bitfl0wer polyphony updates <h1>Getting started with the Polyphony Client</h1> Wed, 06 Mar 2024 22:42:28 +0000
- Porting chorus to WebAssembly + Client Update bitfl0wer chorus polyphony updates <h1>Porting chorus to WebAssembly + Client Update</h1><p>What the current state of GUI libraries in Rust means for Polyphony and chorus, and why we are porting chorus to WebAssembly.</p> Wed, 06 Mar 2024 22:42:28 +0000
- X.509 in polyproto bitfl0wer X.509 polyproto <h1>Certificates, please: X.509 in polyproto</h1><p>This blog post covers a bit about how and why X.509 is used in polyproto, and how we try to makethe process of implementing your own server and incorporating it into an existing network a littleeasier.</p> Tue, 20 Feb 2024 10:10:12 +0000
- Account migration in polyproto bitfl0wer polyproto <p>Account migration is an important and difficult thing to get right in federated systems. In this blogpost, I will outline how I imagine account migration to work in polyproto, and what benefits thisapproach brings.</p> Thu, 08 Feb 2024 16:39:01 +0000
\ No newline at end of file
diff --git a/search/search_index.json b/search/search_index.json
index 4fa6a631..91d457eb 100644
--- a/search/search_index.json
+++ b/search/search_index.json
@@ -1 +1 @@
-{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Polyphony","text":"
This documentation currently hosts information about polyproto; an advanced, secure and scalable protocol for federation that empowers users.
The core of the protocol lies in the polyproto-core specification and in the federation API routes, which are used to negotiate and establish connections between foreign servers and clients.
"},{"location":"#glossary","title":"Glossary","text":" - Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/","title":"APIs","text":""},{"location":"APIs/MLS/","title":"polyproto-mls","text":" - Core Routes: Registration needed
- Core Routes: No registration needed
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/","title":"MLS Routes: No registration needed","text":"TODO
This is a work in progress. MLS-related content is currently being migrated over from the polyproto-core specification. This document is not yet complete. Feel free to contribute by opening a pull request on the docs repository. The API will change (possibly drastically) in the future.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#encryption","title":"Encryption","text":"Client-Foreign Server API endpoints concerned with encryption related tasks.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#get-keypackages","title":"GET KeyPackage(s)","text":"/.p2/core/v1/keypackage/:fid
Request KeyPackages - initial encryption keying material - for a specific actor from the server. The requested actor must be registered on this server.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#request","title":"Request","text":""},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#parameters","title":"Parameters","text":"Name Type Description fid String, Federation ID The Federation ID of the actor whose KeyPackage(s) should be returned."},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#body","title":"Body","text":"This request has no body.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#response","title":"Response","text":"200 OK"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#body_1","title":"Body","text":"Type Description JSON-Array of KeyPackage(s), Base64 The actor's KeyPackage(s), Base64 encoded. Each entry in the array corresponds to a different client the requested actor is authenticated on. JSON
[...]\n
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/","title":"MLS Routes: Registration needed","text":"
TODO
This is a work in progress. MLS-related content is currently being migrated over from the polyproto-core specification. This document is not yet complete. Feel free to contribute by opening a pull request on the docs repository. The API will change (possibly drastically) in the future.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#encryption","title":"Encryption","text":"
Client-Home Server API endpoints concerned with encryption, such as KeyPackage management.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#post-add-keypackage","title":"POST Add KeyPackage","text":"
/.p2/core/v1/keypackage/@me
Add a KeyPackage to your KeyPackage store on the server. Only adds KeyPackages to the ID-Cert corresponding to the session token used in the authorization-Header.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#request","title":"Request","text":""},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body","title":"Body","text":"Type Description JSON-Array of KeyPackages One or more KeyPackages to add to the available KeyPackages for this actor. JSON
[ {...}, {...} ]\n
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#response","title":"Response","text":"201 Created"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body_1","title":"Body","text":"
This response has no body.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#put-replace-last-resort-keypackage","title":"PUT Replace Last Resort KeyPackage","text":"
/.p2/core/v1/keypackage_lr
Replace a Last Resort KeyPackage with a new one. Only manipulates Last Resort KeyPackages for the ID-Cert corresponding to the session token used in the authorization-Header.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#request_1","title":"Request","text":""},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body_2","title":"Body","text":"Type Description KeyPackage The KeyPackage to replace the current Last Resort KeyPackage with. JSON
{...}\n
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#response_1","title":"Response","text":"204 No Content"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body_3","title":"Body","text":"
This response has no body.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/auth/","title":"polyproto-auth","text":"
- Authentication Routes: Registration needed
- Authentication Routes: No registration needed
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/","title":"Authentication Routes: No registration needed","text":"
All API endpoints needed for implementing polyproto-auth. This Page only includes routes, for which a client does not need a \"Client-Home Server relationship\" with the server.
Unfinished section
TODO: This section is not yet finished. It is missing descriptions for most routes, some error-code responses, etc.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#authorization","title":"Authorization","text":"
Bearer token, unless specified otherwise.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#errors","title":"Errors","text":"
The errors listed below are not exhaustive, and only include the most common errors associated with an endpoint. For rate limit errors, see the Rate Limits documentation.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#post-create-identity","title":"POST Create Identity","text":"
/.p2/core/v1/register
Creates an identity on a given server.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#request","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body","title":"Body","text":"
TODO: Re-evaluate if auth_payload is needed here.
Name Type Description
actor_name String The preferred name for this new identity.
auth_payload JSON-Object n. A. JSON
{\n \"actor_name\": \"alice\",\n \"auth_payload\": {\n \"password\": \"3c4589y70masfnmAML2\"\n }\n}\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#response","title":"Response","text":"201 Created409 Conflict Text Only
##### Body\n\n| Name | Type | Description |\n| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------- |\n| `fid` | String | The [Federation ID](../../Glossary.md#federation-id) of the new identity. |\n| `payload` :material-help:{title=\"This field is optional.\"} :material-code-braces:{title=\"The actual contents of this attribute are implementation-specific. polyproto-core does not provide any defaults for this field.\"} | JSON-Object | - |\n\n```json\n{\n \"fid\": \"xenia@example.com\",\n \"payload\": {\n \"some_account_information\": \"important information\",\n \"is_awesome\": true\n }\n}\n```\n
Text Only
Returned when the requested actor name is already taken within the namespace.\n\n##### Body\n\n```json\n{\n \"errcode\": 409,\n \"error\": \"P2CORE_FEDERATION_ID_TAKEN\"\n}\n```\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#post-identify","title":"POST Identify","text":"
/.p2/core/v1/session/identify
Identify on a foreign server and receive a session token.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#request_1","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_1","title":"Body","text":"Name Type Description
challenge String The completed challenge, consisting of a UTF-8 encoded signature value and the original challenge string. The signature value must have been created using the private identity key of that actor.
id_cert String, PEM, Base64 The client's ID-Cert, encoded in PEM & Base64.
auth_payload JSON-Object - JSON
{\n \"completed_challenge\": {\n \"challenge\": \"UH675678rbnFGNHJV2ijcnr3ghjHV74jah...\",\n \"signature\": \"Ac4hjv2ijcnr3ghjHV74jahUH675678rbnFGNHJV...\"\n },\n \"id_cert\": \"gA3hjv2ijcnr3ghjHV74jahUH675678rbnFGNHJV...\",\n \"auth_payload\": {\n \"my_custom_attribute\": \"my_custom_value\"\n }\n}\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#response_1","title":"Response","text":"201 Created"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_2","title":"Body","text":"Name Type Description
token String A session token, to be used for further identification/authentication
payload JSON-Object - JSON
{\n \"token\": \"G5a6hjv2ijcnr3ghjHV74jahUH675678rbnFGNHJV...\",\n \"payload\": {\n \"my_custom_response_attribute\": \"my_custom_response_value\"\n }\n}\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#put-revoke-session-authentication","title":"PUT Revoke session authentication","text":"
/.p2/core/v1/session/revoke
Revoke the current session's authentication by having the server invalidate the session token. Can also be seen as a \"logout\" operation.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#request_2","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_3","title":"Body","text":"
This request has no body.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#response_2","title":"Response","text":"204 No Content"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_4","title":"Body","text":"
This response has no body.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/","title":"Authentication Routes: Registration needed","text":"
All API endpoints needed for implementing polyproto-auth. This Page only includes routes which a client can request from its home server. For routes which can also be accessed from a foreign server, or with no authentication at all, see the Client-Foreign Server API documentation
Unfinished section
TODO: This section is not yet finished. It is missing descriptions for most routes, some error-code responses, etc.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#authorization","title":"Authorization","text":"
Bearer token, unless specified otherwise.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#errors","title":"Errors","text":"
The errors listed below are not exhaustive, and only include the most common errors associated with an endpoint. For rate limit errors, see the Rate Limits documentation.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#post-authenticate-new-session","title":"POST Authenticate new Session","text":"
/.p2/core/v1/session/trust
Creates a new id_cert and a session token from a csr.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#request","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#body","title":"Body","text":"Name Type Description
actor_name String The actor name of the identity to authenticate as.
csr String, PEM A certificate signing request (CSR)
auth_payload JSON-Object n. A."},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#response","title":"Response","text":"201 Created"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#body_1","title":"Body","text":"Name Type Description
id_cert String, PEM The ID-Cert for this unique Identity-Session combination
token String An authorization secret, called a \"token\", valid for this
id_cert."},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Overviews/core/","title":"An Overview of polyproto","text":"
Work in Progress
This overview page is not yet finished. However, that what is there is already representative of what the polyproto protocol is about.
polyproto is a federated identity and message exchange protocol, which can be used for almost anything. If you'd like to build an application where federation, user control and data integrity are wanted, then polyproto is most likely for you. Read this overview to get to know the core concepts and technologies used in the protocol.
"},{"location":"Overviews/core/#identity","title":"Identity","text":"
Your identity is always represented by a Federation ID, FID for short. Conceptually, FIDs are nothing new, and they look like this:
xenia@some.example.com
Everything after the @ is your Home Servers' domain, and the part before the @ is your username. Together, this makes for an individual, yet globally unique identifier.
"},{"location":"Overviews/core/#certificates-and-keys","title":"Certificates and Keys","text":"
Identity Certificates - ID-Certs for short - represent your identity when logged in on different sessions. Each Identity Certificate contains the following information:
- Your federation ID, so that an account can be uniquely identified
- A session ID, which is unique for each session and does not change, even if the keys change
- An expiry date, after which the certificate becomes invalid
- A signature, generated by your home server, which acts as part of the proof that this certificate was actually issued by your home server
- Some information from your home server (Home server domain, certificate serial number)
- Information about the signature algorithm used
and, last but not least
- The public identity key of the client
For the sake of explanation, the most important parts here are the client public identity key, your federation ID, the home servers' domain and the home servers' signature for this certificate.
"},{"location":"Overviews/core/#message-signing","title":"Message signing","text":"
When you, for example, chat with someone on a different server, that other server is fully in control about what data it chooses to present to you. To make sure that this server is always telling you the truth, and not, for example, manufacturing chat messages or social media posts made by a person, messages are signed using a clients' public identity key.
flowchart LR\n hs[(Your Home Server)] --- you(You)\n you -- Send signed message --> fs[(A Foreign Server)]\n fs -- Forwards message --> other(Other User)\n fs --> verify{Verify message signature}\n other --> verify\n verify -- Get certificate from home server to verify --> hs
This is how it works:
- As touched on previously, every user client has an own identity key pair, comprised of a public and a private key. The public key is cryptographically linked to the private key, meaning that this public key can not belong to another private key. Signing data is done using the private key, which ONLY the client knows. Everyone can then use your public key to prove that this signature was generated by your client, and that the signature matches the data which was signed.
- Signatures are unique to a piece of data, meaning that two differing pieces of data signed by the same or different private keys will always1 produce different signatures. This is the case, even if the data only differs minutely (be it by a single space, or a single comma).
- Your home server attests to a clients' key pair, by creating a certificate for your public key, which it signs with its own secret, public/private key pair, and then sends to you. Your private key is never sent anywhere at all, and it does not need to be.
Now, your public identity key and your home servers' identity key are 'linked' to each other. This is represented in the ID-Cert you then receive from your home server.
- When communicating with another \"foreign\" server in polyproto, you first send that server your ID-Cert. The server can then prove the validity of your identity, simply by asking your home server for its public key and performing a quick signature verification.
- When sending data to the server, such as chat messages, your client computes the signature for that message using your private key, and attaches this signature to the message you send to other servers.
- Any user, at any point, can now take this signature, your identity certificate and your home servers' public key and cryptographically verify that it was, in fact, you who sent the message, and that the message was not tampered with in any way. To distribute the load of ID-Cert requests more evenly, it is always the duty of the server that the data exchange is happening on, to cache and hand out ID-Certs of users.
Info
If you are interested about the details, feel free to jump to section 7.1 in the protocol in the specification document, which covers this exact thing and more.
"},{"location":"Overviews/core/#trust","title":"Trust","text":"
Trusting the smallest possible amount of entities is great practice when it comes to security. polyproto makes sure that almost everyone you do trust is under constant scrutiny, and thus provides measures to verify a data authors' identity, and that the actual data has not been tampered with.
Aside from yourself, the entity with the most trust assigned to it is your home server. Creating your identity on a specific home server is a pledge from that server and its admins to you, where they promise not to create sessions on your behalf, or to otherwise perform actions which can be publicly identified to be carried out by you, without your explicit consent.
Should you ever change your mind about your home server's trustworthiness, you can always migrate to another server while keeping the ownership status of your data on all servers you have sent data to, even if your home server is offline indefinitely.
"},{"location":"Overviews/core/#multi-use","title":"Multi-use","text":"
polyprotos' API definitions and specification document intentionally leave space for implementation-specific data to be sent, where it makes sense. Nothing about the core protocol makes polyproto inherently unsuitable for any purpose.
"},{"location":"Overviews/core/#federation","title":"Federation","text":"
Federation in polyproto means using one identity or client to interact with multiple servers or even services at once. Implementing federation is straightforward, and entirely seamless to use for end users.
"},{"location":"Overviews/core/#technology","title":"Technology","text":"
Probably the most refreshing aspect about this new protocol is, that it is really boring. There is really nothing new about any given atomic aspect of polyproto. Polyproto uses well-known, tried and battle-tested technologies, such as asymmetric encryption, X.509-based public key infrastructure and -certificates, digital signatures, JSON over REST and other, already well established technologies such as WebSockets.
polyproto should be effortless - both for developers and for end users, who, ideally, should never have to notice any of the technical stuff going on in the background.
"},{"location":"Overviews/core/#conclusion","title":"Conclusion","text":"
This is just an outline about how polyproto works. The goal with this outline is to inform about the most relevant parts, while intentionally leaving out some details for the sake of clarity. If you have read and understood this overview, you should have no - or at least way less - trouble reading the full protocol specification, which covers a lot more details!
-
Signature/hash collisions, which although theoretically possible, are extraordinarily infrequent and thus, negligible in practical scenarios.\u00a0\u21a9
"},{"location":"Protocol%20Specifications/core/","title":"polyproto Specification","text":"
v0.1.0-alpha.1 - Treat this as an unfinished draft.
Semantic versioning v2.0.0 is used to version this specification. The version number specified here also applies to the API documentation.
- polyproto Specification
- 1. Terminology used in this document
- 2. Trust model
- 3. APIs and communication protocols
- 3.3 WebSockets
- 3.3.1 Events over REST
- 4. Federated identity
- 4.1 Authentication
- 4.1.1 Authenticating on a foreign server
- 4.1.2 Sensitive actions
- 4.2 Challenge strings
- 4.3 Protection against misuse by malicious home servers
- 5. Federation IDs (FIDs)
- 6. Cryptography and ID-Certs
- 6.1 Home server signed certificates for public client identity keys (ID-Cert)
- 6.1.1 Structure of an ID-Cert
- 6.1.1.1 Identity Descriptors (IDDs)
- 6.1.1.2 Extensions and constraints
- 6.1.1.3 Session IDs
- 6.1.2 Necessity of ID-Certs
- 6.1.3 Key rotation
- 6.1.4 Early revocation of ID-Certs
- 6.2 Actor identity keys and message signing
- 6.2.1 Message verification
- 6.2.2 Handling of external messages
- 6.3 Private key loss prevention and private key recovery
- 6.4 Cryptographic recommendations
- 6.5 Best practices
- 6.5.1 Signing keys and ID-Certs
- 6.5.2 Home server operation and design
- 6.5.3 Private key loss prevention and private key recovery
- 7. Migrations
- 7.1 Identity migration
- 7.1.1 Redirects
- 7.2 Re-signing messages
- 7.2.1 Message batches
- 7.2.2 Server imposed limits
- 7.2.2.1 Body size
- 7.2.2.2 Interval between re-signing batches
- 7.3 Moving data
- 7.3.1 Content Addressing with relative roots
- 7.4 Challenges and trust
- 8. Protocol extensions (P2 extensions)
- 8.1 Extension design
- 8.2 Namespaces
- 8.3 Officially endorsed extensions
- 8.4 Versioning and yanking
- 8.4.1 Yanking
- 8.5 Dependencies
- 8.6 Routes
- 9. Services
- 9.1 Discoverability
- 9.1.1 Changing a primary service provider
The polyproto protocol is a home-server-based identity federation protocol specification intended for use in applications where actor identity is needed. polyproto focuses on federated identity, and does not specify any further application-specific features. It can be used standalone, as a method of authenticating across many applications and services, or as a base for federated protocol extensions and application implementations. The use of cryptography - namely digital signatures and X.509 certificates - make polyproto identities verifiable and portable. polyproto empowers actors, as the home server can be changed at any time, without losing data or connections to other actors.
This document is intended to be used as a starting point for developers wanting to develop software, which can operate with other polyproto implementations.
"},{"location":"Protocol%20Specifications/core/#1-terminology-used-in-this-document","title":"1. Terminology used in this document","text":"
The following terminology is used throughout this document:
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Protocol%20Specifications/core/#2-trust-model","title":"2. Trust model","text":"
polyproto operates under the following trust assumptions:
- Users entrust their home server and its admins with data security and discretion on actions appearing as actor-performed as, as with most home server based systems, it is possible for a home server to impersonate an actor in unencrypted communications.
- Impersonation can be detected by users, as home servers never have access to private keys of actors. To sign messages as an actor, a home server would have to use a different key pair.
- Users only trust information, which can be verified by cryptographic means. This includes verifying the identity of other actors and verifying the integrity of messages.
- In a federated context, users trust foreign servers with all unencrypted data they send to them.
- Foreign servers cannot impersonate users without immediate detection. Outsiders, meaning foreign servers and other actors, are unable to produce signatures that have a cryptographic connection to the actors' home server. This is assuming correct implementation of cryptographic standards, secure home server operation and non-compromised client devices, all of which are mostly out of the scope of this specification.
- Users rely on their home server for identity key certification, without the home server possessing the identity.
"},{"location":"Protocol%20Specifications/core/#3-apis-and-communication-protocols","title":"3. APIs and communication protocols","text":"
The polyproto specification defines a set of APIs. In addition to these REST APIs, polyproto employs WebSockets for real-time communication between clients and servers.
The APIs are divided into two categories:
- Routes: No registration needed: These routes are available to all clients, regardless of whether this server is the client's home server.
- Routes: Registration needed: These routes are only available to clients where the server is the client's home server.
All software aiming to federate with other polyproto implementations must implement the APIs defined in this specification. Implementations can choose to extend the APIs with additional routes, but must not remove or change the behavior of the routes defined in this specification.
"},{"location":"Protocol%20Specifications/core/#33-websockets","title":"3.3 WebSockets","text":"
WebSockets enable real-time communication between actor clients and servers.
WebSocket connections to polyproto servers consist of the following cycle:
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant g as Gateway\n\nc->>g: Establish connection\ng->>c: Recieve hello event\n\nloop TODO: interval\n c->>g: Send heartbeat event\n g->>c: Send heartbeat ACK Event\nend\n\nc->>g: Send identify payload\n\nalt Server accepts\n g->>c: Send ready event\nelse Server defined reason\n g->>c: Disconnect with specified reason\nend\n\n\nopt Resume connection#59;<br />otherwise, repeat from step 1\n c->>g: Open new connection\n c->>g: Send resume event\n g->>c: Send missed events\n g->>c: Send resumed event\nend\n
Fig. 1: Sequence diagram of a WebSocket connection to a polyproto server.
Info
To learn more about polyproto WebSockets and WebSocket Events, consult the WebSockets documentation.
"},{"location":"Protocol%20Specifications/core/#331-events-over-rest","title":"3.3.1 Events over REST","text":"
For some implementation contexts, a constant WebSocket connection might not be wanted. A client can instead opt to query an API endpoint to receive events, which would normally be sent through the WebSocket connection. Concrete polyproto-implementations and extensions can decide whether this alternative behavior is supported.
Example
An example of an implementation context where having a constant WebSocket might not be wanted would be Urban IoT devices, or devices with a limited or only periodically available internet connection.
Querying this endpoint yields a JSON-Array containing all events the session has missed since last querying the endpoint, or since last being connected to the WebSocket.
Depending on how many events the session has missed, the earliest events might be excluded from the response to limit the response bodies size. This behavior should be explicitly documented in implementations or extensions of polyproto.
Due to the intended use cases for retrieving events through REST rather than WebSockets, this endpoint is not a long-polling endpoint.
There are three intended, main modes for retrieving events in polyproto
- Keep a constant WebSocket connection whenever possible
- Keep a semi-constant WebSocket connection, perhaps connecting every x minutes for a set period of time
- Do not use WebSockets and only query the REST API
Polling a REST endpoint is inherently inefficient and therefore should only be done with a high interval, ranging from a few minutes to a few days. If a client requires information more often than that, then a WebSocket connection should be considered.
"},{"location":"Protocol%20Specifications/core/#4-federated-identity","title":"4. Federated identity","text":"
The federation of actor identities allows users to engage with foreign servers as if they were their home servers. For example, in polyproto-chat, an actor can send direct messages to users from a different server or join the Guilds of other servers.
Identity certificates defined in sections #6. Cryptography and ID-Certs and #6.1 Home server signed certificates for public client identity keys (ID-Cert) are employed to sign messages that the actor sends to other servers.
Using one identity for several polyproto implementations
An actor can choose to use the same identity for multiple polyproto implementations. Read section #9 for more information.
Info
You can read more about Identity Certificates in section #6.
"},{"location":"Protocol%20Specifications/core/#41-authentication","title":"4.1 Authentication","text":"
The core polyproto specification does not contain a strict definition of authentication procedures and endpoints. This allows for a wide range of authentication methods to be used. However, if implementations want to closely interoperate with each other, they should highly consider implementing the polyproto-auth standard for authenticating on home servers and foreign servers alike.
Warning
Close interoperation is only possible if all involved polyproto implementations have an overlapping set of supported authentication methods. Therefore, it is highly recommended to implement and use the polyproto-auth standard, unless your use case requires strictly requires a different authentication method. Of course, other authentication methods can be implemented in addition to polyproto-auth.
When successfully authenticated, a client receives a session token, which can then be used to access authenticated routes on the REST API and to establish a WebSocket connection. Each ID-Cert can only have one active session token at a time.
About session tokens
Session tokens are used to authenticate a user over a longer period of time, instead of, for example, requiring the user to solve a challenge string every time they want to access a protected route.
"},{"location":"Protocol%20Specifications/core/#411-authenticating-on-a-foreign-server","title":"4.1.1 Authenticating on a foreign server","text":"
Regardless of the authentication method used, the foreign server must verify the actor's identity before allowing them to perform any actions. This verification must be done by proving the cryptographic connection between an actors' home servers' public identity key and the actors' ID-Cert. Challenge strings, as described in Section 4.2 and in polyproto-auth are used for this purpose within this specification.
Servers must also check with the actors' home server to ensure that the ID-Cert has not been revoked. APIs for this purpose are defined in the API documentation.
"},{"location":"Protocol%20Specifications/core/#412-sensitive-actions","title":"4.1.2 Sensitive actions","text":"
Warning
Sensitive actions should require a second factor of authentication, apart from the actors' private key. This second factor can be anything from a password to TOTP or hardware keys, depending on the authentication method or standard used.
If this is not done, a malicious user who gained access to an actors' private key can lock that actor out of their account entirely, as the malicious user could revoke the actors' other ID-Certs, and thus prevent the actor from logging in again.
Sensitive actions include, but are not limited to:
- Generating a new ID-Cert
- Revoking an ID-Cert
- Changing the actors' federation ID
- Changing the actors' other factors of authentication
- Server administration actions
Clients should be prepared to gracefully handle the case where a sensitive action fails due to a lack of a second factor of authentication, and should prompt the user to provide the second factor of authentication.
"},{"location":"Protocol%20Specifications/core/#42-challenge-strings","title":"4.2 Challenge strings","text":"
Servers use challenge strings to verify an actor's private identity key possession, without revealing the private key itself. These strings, ranging from 32 to 256 UTF-8 characters, have a UNIX timestamp lifetime. If the current timestamp surpasses this lifetime, the challenge fails. The actor signs the string, sending the signature and their ID-Cert to the server, which then verifies the signature's authenticity.
All challenge strings and their responses created must be made public to ensure that a chain of trust can be maintained. A third party should be able to verify that the challenge string, which authorized a specific change in data, was signed by the correct private key. The API routes needed to verify challenges as an outsider are documented in the API documentation.
Tip
For public-facing polyproto implementations, it is recommended to use a challenge string length of at least 64 characters, including at least one character from each of the alphanumeric character classes ([a-zA-Z0-9]). Server implementations should ensure that challenge strings are unique per actor. If this is not the case, actors could potentially be the target of replay attacks.
Challenge strings can counteract replay attacks. Their uniqueness ensures that even identical requests have different signatures, preventing malicious servers from successfully replaying requests.
"},{"location":"Protocol%20Specifications/core/#43-protection-against-misuse-by-malicious-home-servers","title":"4.3 Protection against misuse by malicious home servers","text":"
To protect users from misuse by malicious home servers, a mechanism is needed to prevent home servers from generating federation tokens for users without their consent and knowledge.
Potential misuse scenario
A malicious home server can potentially request a federation token on behalf of one of its users, and use it to generate a session token on the actor's behalf. The malicious server can then impersonate the actor on another server, as well as read unencrypted data (such as messages, in the context of a chat application) sent on the other server.
Abstract
The above scenario is not unique to polyproto, and rather a problem other federated services/protocols, like ActivityPub, have as well. There is no real solution to this problem, but it can be mitigated a bit by making it more difficult for malicious home servers to do something like this without the actor noticing.
Polyproto servers need to inform users of new sessions. This visibility hampers malicious home servers, but does not solve the issue of them being able to create federation tokens for servers the actor does not connect to. This is because, naturally, users cannot receive notifications without a connection. Clients re-establishing server connections must be updated on any new sessions generated during their absence. The NEW_SESSION gateway event must be dispatched to all sessions, excluding the new session. The NEW_SESSION event's stored data can be accessed in the Gateway Events documentation.
Note
With proper safety precautions and strong encryption, it is extremely unlikely for a malicious server to be able to listen in on encrypted conversations, without all users in that conversation noticing. When implementing the polyproto-mls P2 extension, MLS's forward secrecy guarantees ensure that, in theory, a malicious session cannot decrypt any messages sent before its' join epoch. If secrecy or confidentiality are of concern, users should host their own home server and use end-to-end encryption, such as polyproto-mls.
"},{"location":"Protocol%20Specifications/core/#5-federation-ids-fids","title":"5. Federation IDs (FIDs)","text":"
Every client requires an associated actor identity. Actors are distinguished by a unique federation ID (FID). FIDs consist of their identifier, which is unique per instance, and the instance's root domain. This combination ensures global uniqueness.
FIDs used in public contexts are formatted as actor@optionalsubdomain.domain.tld, and are case-insensitive.
The following regular expression can be used to validate actor IDs: \\b([a-z0-9._%+-]+)@([a-z0-9-]+(\\.[a-z0-9-]+)*).
Info
The above regular expression is flavored for the Rust Programming Language, but can be easily adapted to other languages.
Note
Validating a federation ID with the above regex does not guarantee that the ID is valid. It only indicates that the federation ID is formatted correctly.
For all intents and purposes, a federation ID is a display of identity. However, verifying identity claims is crucial. See Section #6.1 and Section #6.2.2 for more information.
"},{"location":"Protocol%20Specifications/core/#6-cryptography-and-id-certs","title":"6. Cryptography and ID-Certs","text":""},{"location":"Protocol%20Specifications/core/#61-home-server-signed-certificates-for-public-client-identity-keys-id-cert","title":"6.1 Home server signed certificates for public client identity keys (ID-Cert)","text":"
The ID-Cert, a X.509 certificate, validates a public actor identity key. It is an actor-generated CSR (Certificate Signing Request), signed by a home server, encompassing actor identity information and the client's public identity key. Clients can get an ID-Cert in return for a valid and well-formed CSR. Generating a new ID-Cert is considered a sensitive action and therefore should require a second factor of authentication.
A CSR in the context of polyproto will be referred to as an ID-CSR. ID-CSRs are DER- or PEM-encoded PKCS #10 CSRs, with a few additional requirements.
All ID-Certs are valid X.509 v3 certificates. However, not all X.509 v3 certificates are valid ID-Certs.
ID-Certs form the basis of message signing and verification in polyproto. They are used to verify the identity of a client, and to verify the integrity of messages sent by a client.
An ID-CSR includes the following information, according to the X.509 standard:
- The public identity key of the client.
- An identity descriptor (IDD), describing the actor the certificate is issued to. The IDD must be formatted according to Section 6.1.1.1.
- The signature algorithm used to sign the certificate.
- The signature of the certificate, generated by using the entities' private identity key.
When signing an ID-CSR, the home server must verify the correctness of all claims presented in the CSR.
Important
All entities receiving an ID-Cert MUST inspect the certificate for correctness and validity. This includes checking whether the signature matches the certificates' contents and checking the certificate's validity period.
Actors must use a separate ID-Cert for each client or session they use. Separating ID-Certs limits the potential damage a compromised ID-Cert can cause.
For two implementations of polyproto to be interoperable, they must support an overlapping set of digital signature algorithms. See Section 6.4 for more information on cryptographic recommendations.
"},{"location":"Protocol%20Specifications/core/#611-structure-of-an-id-cert","title":"6.1.1 Structure of an ID-Cert","text":"
The ID-Cert is a valid X.509 certificate, and as such, it has a specific structure. The structure of an X.509 certificate is defined in RFC5280. ID-Certs encompass a subset of the structure of an X.509 certificate.
ID-Certs have the following structure:
Field Description Special requirements, if any X.509 equivalent Correctly formatted Name attribute, according to #6.1.1.1 Identity descriptor Issuer Name A unique identifier for the certificate, used by the CA to identify this certificate. Must be unique across all certificates issued by a home server. Serial Number The algorithm used to sign the certificate. Certificate Signature Algorithm & Signature Algorithm ID The signature of the certificate, generated by using the home servers' private identity key. Certificate Signature The expiry date of the certificate. Time must not be after expiry date of the home server's root certificate Validity period: Not After Certificate validity period starting date Time must not be before the home server's root certificate was generated Validity period: Not Before X.509 Version Number (v3) polyproto only uses Version 3 X.509 certificates. Version Number The public identity key of the client. Subject Public Key Info: Subject Public Key The public key algorithm used to generate the client's public identity key. Subject Public Key Info: Public Key Algorithm The session ID of the client. No two valid certificates for one session ID can exist. Session IDs have to be unique per user. Subject Unique Identifier Extensions Extensions and Constraints Extensions"},{"location":"Protocol%20Specifications/core/#6111-identity-descriptors-idds","title":"6.1.1.1 Identity Descriptors (IDDs)","text":"
polyproto Identity Descriptors are a subset of the X.509 certificate's distinguished name. Distinguished Names (DNs), according to the LDAP Data Interchange Format (LDIF). The DN is a sequence of relative distinguished names (RDNs).
The identity descriptor must be unique for each certificate issued by a home server. The DN of an ID-Cert must meet all of the following requirements:
- Identity descriptor (IDD) must have \"common name\" attribute. If the ID-Cert is for an actor, the common name must be the local name of the actor. In the case of an actor with an FID of
xenia@example.com, the \"common name\" would be xenia. If the ID-Cert is a self-signed home server certificate, the \"common name\" attribute must not be present. - Must have at least one domain component, specifying the home servers' FQDN (fully qualified domain name).
- If the ID-Cert or ID-CSR is for an actor, the IDD must include the
UID (OID 0.9.2342.19200300.100.1.1) and uniqueIdentifier (OID 0.9.2342.19200300.100.1.44) fields. UID is the federation ID of the actor, e.g. actor@fqdn-of-home server.example.com.uniqueIdentifier is a Session ID.
- Can have other attributes, if the additional attributes do not conflict with the above requirements. Additional attributes might be ignored by other home servers and other clients, unless specified otherwise in a polyproto extension. Additional attributes, which are not part of a polyproto extension must be non-critical X.509 extensions.
If the home server does not have a subdomain or top level domain, the dc fields for these components should be omitted.
"},{"location":"Protocol%20Specifications/core/#6112-extensions-and-constraints","title":"6.1.1.2 Extensions and constraints","text":"
The following constraints must be met by ID-Certs:
- If the ID-Cert is a root certificate
- It must have the
CA flag set to true. The path length constraint must be present and set to 0. - It must have the
keyCertSign key usage flag set to true.
- If the ID-Cert is an actor certificate
- It must have the
CA flag set to false or omitted. - It must have the
keyCertSign key usage flag set to false or omitted. - It must have the
digitalSignature key usage flag OR contentCommitment flags set to true.
Key Usage Flags and Basic Constraints are critical extensions. Therefore, if any of these X.509 extensions are present, they must be marked as \"critical\". ID-Certs not adhering to this standard must be treated as malformed.
"},{"location":"Protocol%20Specifications/core/#6113-session-ids","title":"6.1.1.3 Session IDs","text":"
The session ID is an ASN.1 Ia5String chosen by the actor requesting the ID-Cert. It is used to uniquely identify a session. The session ID must be unique for each certificate issued to that actor. A session ID can be re-used if the session belonging to that session ID has become invalid. Session ID re-use in this case also applies, when a different ID-Cert wants to use the same session ID, provided that the session ID is not currently in use. If the session ID is currently in use, the actor requesting the ID-Cert must select a different session ID, as session IDs must not be overridden silently.
Session IDs are 1 - 32 characters long and. They can contain any character permitted by the ASN.1 IA5String type.
Session IDs can be used to identify a session across devices, or to detect if a new, perhaps malicious session has been created.
"},{"location":"Protocol%20Specifications/core/#612-necessity-of-id-certs","title":"6.1.2 Necessity of ID-Certs","text":"
The addition of a certificate is necessary to prevent a malicious foreign server from abusing public identity key caching to impersonate an actor. Consider the following example, which employs foreign server public identity key caching, but no home server issued identity key certificates:
Potential misuse scenario
A malicious foreign server B can fake a message from Alice (Home server: Server A) to Bob (Home Server: Server B), by generating a new identity key pair and using it to sign the malicious message. The foreign server then sends that message to Bob, who will then request Alice's public identity key from Server B, who will then send Bob the malicious public identity key. Bob will succeed in verifying the signature of the message, and not notice that the message has been crafted by a malicious server.
The above scenario is not possible with home server issued identity key certificates, as the malicious server cannot generate an identity key pair for Alice, which is signed by Server A.
"},{"location":"Protocol%20Specifications/core/#613-key-rotation","title":"6.1.3 Key rotation","text":"
A session can choose to rotate their ID-Cert at any time. This is done by generating a new identity key pair, using the new private key to generate a new CSR, and sending the new Certificate Signing Request to the home server, along with at least one new KeyPackage and a corresponding 'last resort' KeyPackage, if encryption is offered. The home server will then generate the new ID-Cert, given that the CSR is valid and that the server accepts the creation of new ID-Certs at this time.
Rotating keys is done by using an API route, which requires authorization.
Note
Sessions can request a new ID-Cert for any session of the same actor. Most other, currently existing services also allow for this, as it is a common use case for user to want to, perhaps, log out of devices they no longer use. Depending on your use case, this might be a security concern. Whether and how this risk is mitigated is up to concrete implementations.
Home servers must keep track of the ID-Certs of all users (and their clients) registered on them, and must offer a clients' ID-Cert for a given timestamp on request. This is to ensure messages sent by users, even ones sent a long time ago, can be verified by other servers and their users. This is because the public key of an actor likely changes over time and users must sign all messages they send to servers. Likewise, a client should also keep all of its own ID-Certs stored perpetually, to potentially verify its identity in case of a migration.
Users must hold on to all of their past key pairs, as they might need them to migrate their account in the future. How this is done is specified in section 6.3: Private key loss prevention and private key recovery.
The lifetime of an actor ID-Cert should be limited to a maximum of 60 days. This is to ensure that even in a worst-case scenario, a compromised ID-Cert can only be used for a limited amount of time. The renewal of an ID-Cert is considered a sensitive action and should require a second factor of authentication. A client that has this second factor of authentication stored should renew the ID-Cert of the authenticated actor without further interaction.
Server ID-Certs should be rotated way less often (every 1-3 years). Only rotate a server ID-Cert if it is suspected to be compromised, is lost, or has expired.
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant s as Server\n\nc->>c: Create CSR for own identity key\nc->>s: Request key rotation/CSR signing, CSR attached\ns->>s: Verify validity of claims presented in the CSR\nalt verify success\n s->>s: Create ID-Cert for Client\n s->>c: Respond with ID-Cert\nend
Fig. 2: Sequence diagram depicting the process of a client that uses a CSR to request a new ID-Cert from their home server.
A server identity key's lifetime might come to an early or unexpected end, perhaps due to some sort of leak of the corresponding private key. When this happens, the server should generate a new identity key pair and broadcast the SERVER_KEY_CHANGE gateway event to all clients. Clients must request new ID-Certs through a CSR. Should a client be offline at the time of the key change, it must be informed of the change upon reconnection.
"},{"location":"Protocol%20Specifications/core/#614-early-revocation-of-id-certs","title":"6.1.4 Early revocation of ID-Certs","text":"
A note about CRLs
It is common for systems relying on X.509 certificates for user authentication to use Certificate Revocation Lists (CRLs) to keep track of which certificates are no longer valid. This is done to prevent a user from using a certificate that has been revoked.
CRLs are difficult to implement well, often requiring many resources to keep up to date, and are also not always reliable. OCSP (Online Certificate Status Protocol) is a more modern, reliable and easier to implement alternative. Still, it potentially requires many resources to keep up with demand, while introducing potential privacy concerns.
polyproto inherently mitigates some of the possible misuse of a revoked certificate, as the validity of a certificate is usually checked by many parties. Especially, if the revocation process is initiated by the actor themselves, the actor already lets all servers they are connected to know that the certificate in question is no longer valid.
polyproto does not require the use of CRLs or OCSP.
An ID-Cert can be revoked by the home server or the actor at any time. This can be done for various reasons, such as a suspected leak of the private identity key.
When an ID-Cert is revoked, the server must revoke the session associated with the revoked ID-Cert. Revoking an ID-Cert is considered a sensitive action and therefore should require a second factor of authentication.
Info
The above paragraph is true for both foreign and home servers. The API routes associated with revoking an ID-Cert are the same regardless of the server type.
Revocation detection
For information on how revocation detection is supposed to be handled, concern the excerpt \"Caching ID-Certs and cache TTLs\"
TODO: Write about identifier changing and how to handle that across servers TODO: Perhaps recommend never using more than a specified number of certificates at once to make re-signing easier
"},{"location":"Protocol%20Specifications/core/#62-actor-identity-keys-and-message-signing","title":"6.2 Actor identity keys and message signing","text":"
As briefly mentioned section #4, users must hold on to an identity key pair at all times. This key pair is used to represent an actor's identity and to verify message integrity, by having an actor sign all messages they send with their private identity key. The key pair is generated by the actor. An actor-generated identity key certificate signing request (CSR) is sent to the actor's home server when first connecting to the server with a new session, or when rotating keys. The key is stored in the client's local storage. Upon receiving a new identity key CSR, a home server will sign this CSR and send the resulting ID-Cert to the client. This certificate is proof that the home server attests to the clients key. Read section 6.1 for more information about the certificate.
"},{"location":"Protocol%20Specifications/core/#621-message-verification","title":"6.2.1 Message verification","text":"
To ensure message integrity through signing, clients and servers must verify message signatures. This involves cross-checking the message signature against the sender's ID-Cert and the senders' home server's ID-Cert, while also confirming the validity of the ID-Cert attached to the message and ensuring its public key matches the sender's.
Info
Signature verification must always be \"strict\", meaning that signature schemes producing malleable signatures and weak public keys must be rejected.
Example: Say we have two actors. Alice, who is registered on Server A, and Bob, who is registered on Server B. Alice and Bob are having a conversation on Server B. Given a signed message from Alice, such as Bob would receive from Server B, the process of verifying the signature would look like this:
sequenceDiagram\nautonumber\n\nactor b as Bob\nparticipant sb as Server B\nparticipant sa as Server A\n\nsb->>b: Alice's signed message\nopt Server A's ID-Cert is not cached on Bob's client\n b->>sa: Request Server A ID-Cert\n sa->>b: Server A ID-Cert\nend\nopt Alice's ID-Cert is not cached on Bob's client\n b->>sb: Request Alice's ID-Cert\n opt Alice's ID-Cert is not cached on Server B\n sb->>sa: Request Alice's ID-Cert\n sa->>sb: Alice's ID-Cert\n end\n sb->>b: Alice's ID-Cert\nend\nb->>b: Verify signature of Alice's message (Fig. 4)
Fig. 3: Sequence diagram of a successful message signature verification.
Bob's client and Server B should now cache Server A's and Alice's ID-Certs, to avoid having to request them again.
The TTL (time to live) of these cached items should be relatively short. Recommended values are between one (1) and twelve (12) hours. Cached ID-Certs must be evicted from the cache, after the TTL has expired. Expired cached ID-Certs must not be used for signature verification of new messages, even if the client cannot renew its cache.
Why not select longer lived TTLs for cached ID-Certs?
Suppose that an actors' private identity key is compromised. The actor notices this, and revokes their ID-Cert. If the TTL of cached ID-Certs is too long, the compromised ID-Cert might still be used for signature verification for a long amount of time, even after the ID-Cert has been revoked. This is a problem in the following hypothetical scenario with malicious actor \"Eve\" and victim \"Alice\":
- Alice's private identity key is compromised.
- Malicious actor Eve logs onto Server X, which Alice has never connected to before.
- Alice notices the breach, requesting the revocation of her ID-Cert on all servers she is connected to.
- Server X does not get this revocation message, as Alice does not know about her connection to Server X.
- Eve can now impersonate Alice on Server X, for as long as the TTL of the cached ID-Cert on Server X has not expired.
If the verification fails, Bob's client should try to re-request the key from Server B first. Should the verification fail again, Bob's client can try to request Alice's public identity key and ID-Cert from Server A (Alice's home server). The signature verification process should then be re-tried. Should the verification still not succeed, the message should be treated with extreme caution.
sequenceDiagram\nautonumber\n\nactor b as Bob\nparticipant sb as Server B\nparticipant sa as Server A\n\nb->>b: Verify signature of Alice's message, attempt 1\nalt Verification fails\n b->>sb: Request Alice's ID-Cert\n sb->>b: Alice's ID-Cert\n b->>b: Verify signature of Alice's message, attempt 2\n opt Verification fails again\n b->>sa: Request Alice's ID-Cert\n sa->>b: Alice's ID-Cert\n b->>b: Verify signature of Alice's message, final attempt\n opt Verification is still unsuccessful\n b-->b: Treat Alice's message with extreme caution.\n end\n end\nelse Verification succeeds\n b-->b: Treat Alice's message as verified.\nend
Fig. 4: Sequence diagram showing how message verification should be handled if the first attempt to verify the signature fails.
TODO: IDEA: To keep other servers from not re-requesting the idcert after the ttls has passed, the idcert should have some sort of timestamp that is signed by the original server, so that clients can verify that a server has the most up-to-date idcert cached for a user -flori
Why should Bob's client request Alice's public identity key from Server B first, if Alice's identity lives on Server A?
In the case where alice@server-a.example.com and bob@server-b.example.com are having a conversation where the communications server is any server other than server-a.example.com, Bob should request Alice's ID-Cert from that server first, instead of from server-a.example.com.
For this example, where Alice \"is on\" Server A and Bob \"is on\" Server B:
Bob's client could request Alice's public identity key from Server A, instead of Server B. However, this is discouraged, as it
- Generates unnecessary load on Server A; Doing it this way distributes the load of public identity key requests more fairly, as the server that the message was sent on is the one that has to process the bulk of public identity key requests.
- Would expose unnecessary metadata to Server A; Server A does not need to know who exactly Alice is talking to, and when. Only Server B, Alice and Bob need to know this information. Always requesting the public identity key from Server A might expose this information to Server A.
Clients should only use Server A as a fallback for public identity key verification, if Server B does not respond to the request for Alice's public identity key, or if the verification fails with the public identity key from Server B.
Info
A failed signature verification does not always mean that the message is invalid. It may be that the actor's identity key has changed, and that Server B has not yet received the new public identity key for some reason. However, if the signature cannot be verified at a certain time, this information must be communicated to the actor performing the verification.
"},{"location":"Protocol%20Specifications/core/#622-handling-of-external-messages","title":"6.2.2 Handling of external messages","text":"
In the context of federation with other federation protocols, such as ActivityPub, it is possible for actors to receive messages, which do not have a signature attached to them. If a P2 extension explicitly allows for this, it is possible for a polyproto server to forward such messages to clients. If a P2 extension does not explicitly allow for this, both servers and clients must reject such messages. Clients receiving unexpected external messages should inform the actor about the fact that a server has tried to send them an invalid, possibly malicious message.
Before a polyproto server forwards such a message to clients, it must add an \"external\" property to the message object. If possible in the data format used, this property should be set to a boolean value of true. If the data format does not support boolean values, the property should be set to a string value of true in all lowercase characters. This property must be passed along to the client or clients receiving the message.
If the actor receiving this external message is human, the client must inform the actor that the message is external, and that the message has not been signed by the sender. External messages should be distinguishable from signed messages at first glance.
"},{"location":"Protocol%20Specifications/core/#63-private-key-loss-prevention-and-private-key-recovery","title":"6.3 Private key loss prevention and private key recovery","text":"
As described in previous sections, actors must hold on to their past identity key pairs, should they want or need to migrate their account.
Home servers must offer a way for actors to upload and recover their private identity keys while not having access to the private keys themselves. Private identity keys must be encrypted with strong passphrases and encryption schemes such as AES, before being uploaded to the server. Authenticated actors can download their encrypted private identity keys from the server at any time. All encryption and decryption operations must be done client-side.
If any uncertainty about the availability of the home server exists, clients should regularly download their encrypted private identity keys from the server and store them in a secure location. Ideally, each client should immediately download their encrypted private identity keys from the server after connecting. Clients should never store key backups in an unencrypted manner.
Whether an actor uploads their encrypted private identity keys to the server is their own choice. It is also recommended backing up the encrypted private identity keys in some other secure location.
The APIs for managing encrypted private identity keys are documented in the API documentation.
- Upload encrypted private key material
- Get encrypted private key material
- Delete encrypted private key material
- Get encrypted private key material upload size limit
"},{"location":"Protocol%20Specifications/core/#64-cryptographic-recommendations","title":"6.4 Cryptographic recommendations","text":"
For two implementations of polyproto to be interoperable, they must support an overlapping set of digital signature algorithms.
If technically practical, all implementations of polyproto must support use of the Ed25519 digital signature algorithm for signing messages and generating ID-Certs. The use of the RSA algorithm for digital signatures is heavily discouraged.
"},{"location":"Protocol%20Specifications/core/#65-best-practices","title":"6.5 Best practices","text":"
The following subsections are dedicated to documenting best practices to consider when implementing polyproto.
"},{"location":"Protocol%20Specifications/core/#651-signing-keys-and-id-certs","title":"6.5.1 Signing keys and ID-Certs","text":"
- When a server is asked to generate a new ID-Cert for an actor, it must make sure that the CSR is valid and, if set, has an expiry date less than or equal to the expiry date of the server's own ID-Cert.
- Due to the fact that a
SERVER_KEY_CHANGE gateway event is bound to generate a large amount of traffic, servers should only manually generate a new identity key pair when absolutely necessary and instead select a fitting expiry date interval for their ID-Certs. It might also be a good idea to stagger the sending of SERVER_KEY_CHANGE gateway events, to prevent a server from initiating a DDoS attack on itself. - When a client or server receives the information that an actor's client identity key has been changed, the client/server in question should update their cached ID-Cert for the actor in question, taking into account the session ID of the new identity key pair.
"},{"location":"Protocol%20Specifications/core/#652-home-server-operation-and-design","title":"6.5.2 Home server operation and design","text":"
- Use a caching layer for your home server to handle the potentially large amount of requests for ID-Certs without putting unnecessary strain on the database.
"},{"location":"Protocol%20Specifications/core/#653-private-key-loss-prevention-and-private-key-recovery","title":"6.5.3 Private key loss prevention and private key recovery","text":"
- It is a good idea for home servers to limit the upload size and available upload slots for encrypted private identity keys.
"},{"location":"Protocol%20Specifications/core/#7-migrations","title":"7. Migrations","text":"
polyproto empowers the end-user by defining straightforward mechanisms to change their home server while preserving their identity, moving messages to another server, or both.
Identity migration allows actors to transparently re-assign ownership of their identity and messages to a new identity. This allows actors to switch home servers while not losing ownership of messages sent by them.
Message migration allows actors to move messages from one service-provider to another in a tamper-resistant way. This makes it possible for actors to switch service providers, taking some or all of their messages with them. Which messages can be moved is up to P2 extensions to define, as it might not always be possible to move all messages. Some messages might be tied to a specific context, which is unavailable on the new server.
Example: Information tied to a specific context
In a chat application, there might exist a group chat with a lot of people in it. Moving your messages from this group chat to another server might be impossible, depending on the architecture of the chat application. Typically, the messages in a group chat are stored on the server hosting the group. Moving the messages of one individual from one server to another is not possible in these cases.
Example: Information not necessarily tied to a specific context
Continuing the chat application example, it might very well be possible to move messages written in a private chat between two actors from one server to another. An examplary architecture where this is possible, is where all private messages are stored on the server of the actor who sent the message. Here, an actor can move their messages to another server without any issues.
Migrating an actor always involves reassigning the ownership of all actor-associated data in the distributed network to the new actor. Should the old actor want to additionally move all data from the old home server to another home server, more steps are needed. Account migration is not considered a sensitive action.
This chapter defines behaviors and security mechanisms associated with migrating an actor identity or messages.
"},{"location":"Protocol%20Specifications/core/#71-identity-migration","title":"7.1 Identity migration","text":"
Transferring message ownership from an old to a new account, known as identity migration, necessitates coordination between the two involved accounts.
Identity migration is a process which can be broken down into the following steps:
- Setting up a redirect
- Re-signing data
It is not required that the new account is located on another home server as the old account. Re-signing data and setting up a redirect are both not mandatory steps. It is up to actors to decide to which extent they wish to perform the migration.
"},{"location":"Protocol%20Specifications/core/#711-redirects","title":"7.1.1 Redirects","text":"
Setting up a redirect is an optional step in the identity migration process, helping make the transition from the old account to the new account smoother.
A redirect has to be confirmed by both the redirection source and the redirection target. The redirect is only valid for one specific redirection target. Redirection targets must be valid actors and their home servers must be reachable when the redirect is being set up.
Info
\"Optional\" does not mean that home servers can choose to not implement this feature. Instead, it means that actors can choose to not use this feature.
sequenceDiagram\nautonumber\n\nactor aa as Alice Old (Redirection source)\nparticipant sa as \"Alice Old\" Home Server\nactor ab as Alice New (Redirection target)\n\nNote over aa, ab: These steps may be done in any order<br/>and are not necessarily sequential\npar Verifying redirect intent by passing key trial\n aa->>sa: Request redirect to Alice New\n sa-)sa: Confirm \"Alice New\"<br/>is valid actor by resolving FID \n sa->>aa: List of keys to<br/>verify + challenge string\n aa->>sa: Completed challenge<br/>for each key on the list\n sa->>sa: Set redirect status to<br/>\"confirmed by redirection source\"\nand Notifying the redirection source's home server of the redirection target\n ab->>sa: Request redirect from Alice Old\n sa->>sa: Verify authenticity of Alice New's identity by verifying ID-Cert\n note over sa: Alice New's ID-Cert is determined to be valid\n sa->>ab: Challenge string (See section 4.1.1:<br/>Authenticating on a foreign server)\n ab->>sa: Completed challenge\n sa->>sa: Set redirect status to<br/>\"confirmed by redirection target\"\nend\nsa->>sa: Check, if both redirection source and target have confirmed the redirect\nalt If both redirection source and target have confirmed the redirect\n sa->>sa: Use HTTP 307 to redirect all requests for<br/>Alice Old to Alice New\nelse\n Note over sa: Do nothing\nend
Fig. 5: Sequence diagram depicting the setting up of a redirect.
Until a redirection source actor deletes their account, the home server of that actor should respond with 307 Temporary Redirect to requests for information about the redirection source. After the redirection source deletes their account, Server A can select to either respond with 308 Permanent Redirect, or to remove the redirect entirely.
"},{"location":"Protocol%20Specifications/core/#72-re-signing-messages","title":"7.2 Re-signing messages","text":"
Re-signing messages is the process of transparently changing the signature of messages while leaving the content of the messages unchanged. \"Transparently\" refers to the fact that an outsider can verify the following facts:
- Both involved actors have agreed to the re-signing of the messages
- The \"old\" actor has proven ownership of the signature keys used to produce the \"old\" signatures of the messages
- The message content has not changed during the re-signing process
The intended use cases for re-signing messages are:
- Changing ownership of messages from one actor to another. This enables seamless transitions between accounts, while preserving the integrity of the messages.
- Reducing the amount of keys that need to be remembered by an actor, done if the actor deems it to be convenient.
- \"Rotate keys of past messages\" - This is useful when an actor's private identity key has been compromised, and the actor wants to ensure that all messages sent by them are still owned by them and not at risk of being tampered with.
Actors must not be able to re-sign messages, to which they cannot prove signature-key ownership of.
Additionally, servers must verify the following things about re-signed messages:
- The new signature matches the messages' contents, and is valid
- The ID-Cert corresponding to the new signature is a valid ID-Cert, issued by the correct home server
- The ID-Cert corresponding to the new signature has a public key that was specified in the
allowedResigningKeys property sent to the server when message re-signing was requested. - The contents of the message have not been changed during the re-signing process
- The
expires UNIX timestamp, specified when the server replied to the re-signing request, has not been reached or passed when the re-signed message was received by the server.
Below is a sequence diagram depicting a typical re-signing process, which transfers ownership of messages from Alice A to Alice B.
sequenceDiagram\nautonumber\n\nactor aa as Alice A\nactor ab as Alice B\nparticipant sc as Server \"C\" with stored<br/>messages from Alice A\n\naa->>sc: Request allow message re-signing for Alice B + list of \"allowed\" pubkeys\nsc->>aa: List of keys to verify + challenge string (Key trial)\naa->>sc: Completed challenge for each key on the list\nsc->>sc: Verify challenge, unlock re-signing for Alice B (only \"allowed\" pubkeys)\nsc->>aa: Re-signing of messages for Alice B allowed\nloop Do, while there are messages left to be re-signed\n ab->>sc: Request message re-signing<br/>for Alice A's messages\n sc->>ab: Batch of old messages,<br/>including the signatures + actor certificates\n Note over ab: The client should fetch missing information<br/>such as missing ID-Certs or server public keys<br/>needed to validate the messages from the<br/>corresponding servers, if applicable\n ab->>ab: Verify that Server C has not<br/>tampered with messages by<br/>checking old signatures with own keys\n ab->>ab: Re-sign messages with own keys\n ab->>sc: Send new messages\n sc->>sc: Verify that only FID and signature related fields have changed\n sc->>sc: Verify that key used to produce signature is on \"allowed\" list\n sc->>ab: Acknowledge successful re-signing of batch\n opt\n ab--)ab: Pause for arbitrary amount of time\n end\nend
To allow for a singular set of behaviors, which fit the three intended use cases mentioned prior, not all messages stored by the server of an actor need to be re-signed. Besides querying for all non-re-signed messages, actors can also query or all non-resigned messages, whose signatures correspond to a specific ID-Cert or set of ID-Certs. The API routes for re-signing messages are documented in the API documentation.
"},{"location":"Protocol%20Specifications/core/#721-message-batches","title":"7.2.1 Message batches","text":"
Messages, which have not yet been re-signed are being delivered to an actor in batches. A batch is a JSON object, representing messages sent, using the same ID-Cert. An exemplary array of message batches, as returned by the server, might look as follows:
JSON
[\n {\n id_cert: \"QLASDiohs79034sjkldfny8eppqxncp7n4g9vozeyuiwofxb...\",\n messages: [\n {\n signature: \"ASDiohs79034sjkldfny8eppqxncp7n4g9vozeyuiwofxb...\",\n content: {\n message: \"Hello!\"\n }\n },\n {\n signature: \"ASDiohs7902347sjkldfny8eafhjhjafdlk4g121ghjkz...\",\n content: {\n message: \"Hello again!\"\n }\n }\n ]\n },\n {\n id_cert: \"QLAxiohs79034sjkldfny8eppqxncp7n4g9vozeyuiwofxn...\",\n messages: [\n {\n ...\n }\n ]\n }\n]\n
The concrete values held by a message batch are up to the concrete implementation. The prior JSON array depicting an array of message batches is only an example. However, it is mandatory that a message batch holds the following information:
- The ID-Cert used to sign the messages in the batch
- An array of messages, which must at least contain the following information:
- The signature of the message
- The full content of the message
Returning re-signed messages to the server is done in the same format as the server sends them to the client.
"},{"location":"Protocol%20Specifications/core/#722-server-imposed-limits","title":"7.2.2 Server imposed limits","text":""},{"location":"Protocol%20Specifications/core/#7221-body-size","title":"7.2.2.1 Body size","text":"
Servers can limit the size of an HTTP request body containing re-signed messages. If a body size limit is imposed, the server must communicate this to clients in their response to a query for messages, which have not yet been re-signed. Communicating the body size limit is done by adding a X-P2-Return-Body-Size-Limit header to the response. If this header is not present or has a value of -1, clients should assume that there is no body-size limit.
"},{"location":"Protocol%20Specifications/core/#7222-interval-between-re-signing-batches","title":"7.2.2.2 Interval between re-signing batches","text":"
Servers can define an interval, which a client must wait for before sending a new batch of re-signed messages to the server.
The server communicates this interval to the client as a response to receiving a batch of re-signed messages from the client. The interval is communicated by adding a X-P2-Wait-Until header to the response. The value of this header is a 64-bit integer. The integer represents a UNIX timestamp, which in turn represents the time, at which the client is allowed to send the next batch of re-signed messages.
Clients should expect that the duration of the interval changes between batches. The server can dynamically adjust the duration, which a client must wait before being allowed to send the next batch of re-signed messages. The server can also select to not impose an interval between re-signing batches. Clients should also expect that the server suddenly decides to impose an interval between re-signing batches, even if it has not done so before.
If this header is not present or has a value of -1, clients should assume that there is no interval between re-signing batches.
Fig. 7: Sequence diagram depicting the re-signing procedure.
"},{"location":"Protocol%20Specifications/core/#73-moving-data","title":"7.3 Moving data","text":"
In cases of an imminent server shutdown or distrust in the old server, moving data from the old server is necessary to prevent data loss. This process extends upon the reassigning ownership process, and usually involves the following steps:
- Using the old account, the client requests a data export from your old home server.
- The old home server sends a data export to the client. The client will check the signatures on the exported data, to ensure it was not tampered with.
- The new account re-signs the data with its own keys and imports it into the new home server.
- The new home server verifies the data and signals that the import was successful.
- The old client requests the deactivation or deletion of the old account on the old home server.
sequenceDiagram\nautonumber\n\nparticipant sa as Server A\nparticipant sb as Server B\nbox Same Device\nactor aa as Alice A\nactor ab as Alice B\nend \n\naa->>sa: Request data export\nsa->>aa: Data export\naa->ab: Data shared on device\nab->>ab: Verify data integrity\nab->>ab: Re-sign data\nab->>sb: Request data import\nsb->>sb: Verify data integrity\nsb->>ab: Data import successful\naa-xsa: Deactivate account
Fig. 8: Sequence diagram depicting the data moving process.
How this process is implemented is up to P2 extensions to define. The above steps are only a guideline. The API routes for data export and import are documented in the API documentation
"},{"location":"Protocol%20Specifications/core/#731-content-addressing-with-relative-roots","title":"7.3.1 Content Addressing with relative roots","text":"
Moving data from one server to another might break references to this data. To prevent this as much as possible, content addressing with relative roots is recommended for data behind an additional layer of indirection.
Example
In a chat service, a user might have posted a message containing a picture. In this example, the picture is stored on the user's home server, which is not necessarily the same server as the chat service. If the user moves their account to another server, the picture might not be accessible anymore.
Content addressing with relative roots aids in preventing this issue. Instead of referring to the absolute URL of the content, the server processing the content generates a unique identifier. This identifier can be used to retrieve the content from the server. Most importantly, this identifier does not change when the content is moved to another server. If the base domain of the new server is known, the identifier can be used to retrieve the content from the new server. The \"relative root\" is the base domain of the server, which is used to retrieve the content.
The uniqueness constraint of the identifier is important. If a collision occurs when trying to move the content to another server, the content cannot be migrated in a way that preserves the references to it. One way to ensure the uniqueness of the identifier is to use a hash function on the content itself. Combining this has with a cryptographically strong nonce, then hashing the result of concatenating the nonce and the hash of the content should yield a unique identifier.
The API route for content addressing with relative roots is formatted as follows:
<server_url>/.p2/core/content/<content_id>
The API route for content addressing with relative roots is documented more thoroughly in the API documentation.
Servers with no need for content addressing with relative roots can select to not implement this feature. Servers not implementing this feature should return a 404 Not Found status code when the API route is accessed. Clients should expect finding servers not implementing this feature.
"},{"location":"Protocol%20Specifications/core/#74-challenges-and-trust","title":"7.4 Challenges and trust","text":"
Changing the publicly visible ownership of actor data requires the chain of trust to be maintained. If an \"old\" account wants to change the publicly visible ownership of its data, the \"old\" account must prove that it possesses the private keys that were used to sign the messages. This is done by signing a challenge string with the private keys. If the server verifies the challenge, it authorizes the new account to re-sign the old account's messages signed with the verified key. Instead of overwriting the message, a new message variant with the new signature is created, preserving the old message.
Implementations and protocol extensions should carefully consider the extent of messages that can be re-signed.
Example
In the case of a social media platform with quote-posting functionality, it is reasonable to assume that re-signing a quoted post is allowed. However, this would likely change the signature of the quoted post, which would be undesirable. Edge cases like these are up to implementations to handle, and should be well documented.
"},{"location":"Protocol%20Specifications/core/#8-protocol-extensions-p2-extensions","title":"8. Protocol extensions (P2 extensions)","text":"
polyproto leaves room for extensions, outsourcing concepts such as concrete message types to protocol extensions. This allows for a more flexible core protocol, which can be adapted to a wide variety of use cases. The following sections define:
- protocol extensions, also called P2 extensions
- how protocol extensions interact with the core protocol
- requirements, which must be fulfilled by protocol extensions to become officially endorsed
"},{"location":"Protocol%20Specifications/core/#81-extension-design","title":"8.1 Extension design","text":"
P2 extensions should be either of the following:
- a major technological addition, which can be taken advantage of by other extensions. Examples of this are:
- a unified WebSocket Gateway connection scheme
- Message Layer Encryption (MLS)
- Compatibility with other protocols (e.g. Matrix, ActivityPub)
- a definition of a service. Examples of this are:
- A federated chat application
- A federated social media platform
A good P2 extension should never be both at the same time. If a P2 extension is both a major technological addition and a document describing a particular application use case, it should likely be split into two separate extensions.
Designing P2 extensions, which only specify a single route or a small set of behavior changes is discouraged. Instead, these should be implemented as part of a larger extension, which offers a more comprehensive set of features.
Note
If you are, say, developing a polyproto server implementation with a feature that is not part of the default polyproto specification, you do not have to create a P2 extension for this feature. P2 extensions are useful for defining interoperable services, which can be implemented by a variety of servers and clients.
"},{"location":"Protocol%20Specifications/core/#82-namespaces","title":"8.2 Namespaces","text":"
A namespace is a string used to identify a specific P2 extension. Used as a prefix in URLs, they prevent route name collisions between different extensions. Namespaces should be unique and descriptive. They must only contain lowercase letters, numbers, hyphens, and underscores. Namespaces must be at least 2 characters long and at most 64 characters long.
Officially endorsed P2 extensions have priority over selecting namespaces. If a namespace is already taken by an officially endorsed extension, a different namespace must be chosen. If a namespace collision exists between an officially endorsed extension and a regular P2 extension, the officially endorsed extension has priority.
"},{"location":"Protocol%20Specifications/core/#83-officially-endorsed-extensions","title":"8.3 Officially endorsed extensions","text":"
Officially endorsed extensions are extensions that either:
- have undergone review and approval by the polyproto maintainers
- have been developed by the maintainers themselves
- have been developed by a third party and are now maintained by the polyproto maintainers
Contact the polyphony-chat maintainers under info@polyphony.chat if you want to have your extension officially endorsed.
Officially endorsed extensions must fulfill all the requirements listed in section 8.
Each version of an extension developed by outside parties must undergo the review process before being officially endorsed.
"},{"location":"Protocol%20Specifications/core/#84-versioning-and-yanking","title":"8.4 Versioning and yanking","text":"
Semantic Versioning v2.0.0 is used for versioning P2 extensions. The version number of an extension is defined in the extension's documentation. The version number must be updated whenever a change is made to the extension. The only exception to this rule is when marking an extension as deprecated (yanking).
"},{"location":"Protocol%20Specifications/core/#841-yanking","title":"8.4.1 Yanking","text":"
Yanking an extension means that the extension is no longer supported, and that it should not be used. A later version of the extension should be used instead. Yanked extension versions should prominently display the \"yanked\" status next to the version number in the extension's documentation.
Versions of officially endorsed P2 extensions can normally not be removed, only marked as yanked.
"},{"location":"Protocol%20Specifications/core/#85-dependencies","title":"8.5 Dependencies","text":"
P2 extensions can depend on other P2 extensions. If an extension depends on another extension, the name of the dependency must be listed in the extension's documentation, along with a link to the dependencies' specification document.
The following syntax is used for indicating the version number of a dependency:
Syntax Meaning
1.0.0 Any version of the dependency with the major version
1, a minor version of
0, and a patch version of
0 or greater is required.
1.0 Any version of the dependency with the major version
1 and the minor version
0 is required. The patch version is unimportant.
1 Any version of the dependency with the major version
1 is required. The minor and patch versions are unimportant.
When selecting a version number for a dependency, the highest possible version number that fulfills the requirements should be selected.
The name of the dependency along with the version number is to be listed right beneath the extension's version declaration in the extension's documentation. Ideally, a link to the dependencies' specification document should be included.
To grow the ecosystem of interoperable services, it is encouraged to first develop a generic version of that service, which acts as a shared base for all implementations. This shared base can then be extended with the exact, non-service-critical features that are needed for a specific implementation.
For example, a generic, federated chat service extension might offer routes for adding reactions to chat messages. However, a route for adding reactions with full-screen animation effects would be better suited as an implementation-specific detail.
If possible for the given use case, P2 extensions should depend on and extend already existing, officially endorsed P2 extensions.
Example
Say, you are developing a social chat platform using polyproto. In this example, you would like your chat platform to have a feature, which is not part of the officially endorsed polyproto-chat extension. Instead of developing a new extension from scratch, your chat extension should likely depend on polyproto-chat, and define only this new feature as part of your own extension.
Doing this ensures a high level of interoperability across all different implementations of a specific application group.
"},{"location":"Protocol%20Specifications/core/#86-routes","title":"8.6 Routes","text":"
Polyproto extensions must never change, add or remove routes defined by the extension they depend on. Instead, routes with alternating or new behavior must be added under a newly defined namespace, which must differ from the original namespace. Changing the behavior of existing routes breaks compatibility with other implementations of the same extension.
Route paths must always start with .p2/, followed by the extensions' namespace. Namespaces are explained in section 8.2.
"},{"location":"Protocol%20Specifications/core/#9-services","title":"9. Services","text":"
Info
A \"service\" is any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services.
Actors can use their identity to register with any server hosting polyproto services, such as polyproto-chat. These servers can be the actors' home server, but can also be foreign servers. There is no limitation to how many services any given actor can register with, and what these services are.
Application specific implementations of polyproto should consider that users of their service might also want to register for services offered by other servers, using the same identity.
"},{"location":"Protocol%20Specifications/core/#91-discoverability","title":"9.1 Discoverability","text":"
The discoverability feature allows users who are registered with the same service but on different servers to communicate with each other. The actor initiating the communication only needs to know the federation ID of the actor they want to communicate with. Consider the following example:
Example: Discovering services
Info
The example below is simplified for the sake of clarity. In a real-world scenario, Alice and the Chat server would perform the foreign server authentication procedure described in section 4.1.1 before Alice can send a chat message to Bob. The example also uses a simplified example of how polyproto-chat works.
Alice and Bob want to communicate with each other. Both Alice and Bob are registered on servers which host the polyproto-chat service. However, Alice and Bob are not registered on the same server, and they do not share any chat rooms. Alice types in Bob's federation ID into her chat client. The client then queries Bob's home server to find out, which server Bob is using for the polyproto-chat service. Alice's client can then send the chat message to Bob's server, which will forward the chat message to Bob.
sequenceDiagram\nautonumber\n\nparticipant sb as Bob's Home Server\nactor aa as Alice\nparticipant sc as Chat server Bob is registered on\nactor ab as Bob\n\naa->>sb: Query: Which server is Bob using for polyproto-chat?\nsb->>aa: Response: URL of Chat server Bob is registered on\naa->>sc: Message to Bob\nsc->>ab: Forward message from Alice to Bob
Fig. 9: Sequence diagram depicting how Alice's client discovers which server Bob is using for the examplary polyproto-chat service.
The example demonstrates how Alice can communicate with Bob, even though they do not share any servers.
To be discoverable, an actor must add a key-value pair to their home server's database. The key is the name of the service, and the value is the base URL of the server hosting the service.
The API routes for managing discoverability are documented in the API documentation
"},{"location":"Protocol%20Specifications/core/#911-changing-a-primary-service-provider","title":"9.1.1 Changing a primary service provider","text":"
Keys are unique in the actor-scoped service->service-provider table. Actors wanting to register for two or more different implementations of the same service must select, which service provider to use as a so-called \"primary service provider\" for that service.
If the actor is human, clients must not override the existing key-value pair silently. Instead, clients must either ask the actor to confirm the change, or not change the key-value pair. Automated actors may override values as they see fit.
Changing a primary service provider entry is considered a sensitive action and should require a second factor of authentication.
Messages do not get moved or re-signed when changing the primary service provider for a given service. If an actor wants to move their messages to the new primary service provider, they must request a migration.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/","title":"P2 Extension: polyproto-auth","text":"
v1.0.0-alpha.1 - Treat this as an unfinished draft. Semantic versioning v2.0.0 is used to version this specification. The version number specified here also applies to the API documentation.
The polyproto-auth extension is a protocol extension for polyproto that provides a basic authentication mechanism to register new users and authenticate existing users.
- P2 Extension: polyproto-auth
- 1. Registration of a new actor
- 1.1 Registering a new actor on a polyproto home server
- 1.2 Authenticating a new client on a polyproto home server
- 1.3 Authenticating on a foreign server
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#1-registration-of-a-new-actor","title":"1. Registration of a new actor","text":"
Registering a new actor in the context of polyproto is done through an API route defined in the polyproto-auth \"No registration needed\" API documentation.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#11-registering-a-new-actor-on-a-polyproto-home-server","title":"1.1 Registering a new actor on a polyproto home server","text":"
To register, the client sends the necessary information to their home server. The server verifies the data, checks username availability, and responds with HTTP 201 and the new identity's federation ID, if successful. However, a session token is not provided until the actor authenticates a client, as detailed in section 1.2.
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant s as Server\n\nc->>s: Registration information\ns->>s: Verify correctness of provided information,<br />check if username is available, etc\n\nalt verification successful\n s->>s: Verify provided CSR\n\n alt CSR okay\n s->>s: Sign CSR\n s->>c: HTTP status code 201, with actor federation ID\n end\nend
Fig. 1: Sequence diagram of a successful identity creation process.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#12-authenticating-a-new-client-on-a-polyproto-home-server","title":"1.2 Authenticating a new client on a polyproto home server","text":"
To access their account from a new device, an actor authenticates the session with their home server by sending authentication information and a certificate signing request (CSR) for the new client. If verified successfully, the server signs the CSR and responds with the newly generated ID-Cert and a session token corresponding to this ID-Cert.
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant s as Server\n\nc->>s: Auth information, CSR\ns->>s: Verify correctness of provided auth information\n\nalt Verified successfully\n s->>s: Verify provided CSR\n alt CSR okay\n s->>s: Sign CSR\n s->>c: HTTP status code 201, ID-Cert + session token\n end\nend\n
Fig. 2: Sequence diagram of a successful client authentication process.
The client is now authenticated and can use the session token and ID-Cert to perform actions on behalf of the actor identified by the ID-Cert.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#13-authenticating-on-a-foreign-server","title":"1.3 Authenticating on a foreign server","text":"
Authenticating on a foreign server requires the actor to sign a challenge string with their private identity key and send it, along with their ID-Cert, to the server. The server then validates the ID-Cert's origin, the challenge string's signature, and the ID-Cert's validity.
If the verification is successful, the foreign server can issue a session token to the actor.
Example: Say that Alice is on server A, and wants to authenticate on Server B, using her existing identity.
Alice's client sends a request to Server B for a challenge string, telling Server B the session ID they are communicating from in the process. Upon receiving a response, Alice signs this challenge string with the correct private key. They then send the signature to Server B. Server B can now verify that it was actually Alice who signed the string, and not a malicious outsider. Server B does this by requesting Alice's ID-Cert, specifically the ID-Cert matching the session ID Alice identified with to Server B. If all goes well, server B will send a newly generated session token back to Alice's client. Alice's client can then authenticate with server B by using this token.
sequenceDiagram\nautonumber\n\nactor a as Alice\nparticipant sb as Server B\nparticipant sa as Server A\n\na->>sb: Challenge string request including current Session ID\nsb->>a: Challenge string\na->>sb: Signed challenge, ID-Cert, optional payload\nsb->>sa: Get Server A Public Certificate\nsa->>sb: Send Public Certificate\nsb->>sb: Verify signature of challenge string\nsb->>a: Session token, optional payload
Fig. 3: Sequence diagram of a successful identity verification.
In the diagram, Alice's \"optional payload\" is extra data that might be requested by servers. This is useful when using a single identity across various polyproto implementations, due to differing information needs. The payload is signed with the actor's private identity key.
Likewise, the \"optional payload\" sent by the server in the above diagram can be used by implementations to send additional information to the client. An example might be initial account information.
Example
Alice currently has a polyproto identity, which she created when signing up for \"https://example.com/chat\". When signing up for this service, she didn't need to provide any additional information on registration. However, when she wants to actor her existing identity to sign up for \"https://example.com/social\", she is asked to provide her email address, which she can provide as the \"optional payload\". The server can then store the email address in its' database, associate it with Alice's identity, and let Alice log in with her existing identity.
If Alice's session token expires, they can repeat this process of requesting a challenge string and, together with her ID-Cert, exchange it for a session token. However, if Alice wants to access this third party account from a completely new device, they will have to perform the steps described in section 1.2 to obtain a valid ID-Cert for that session.
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/","title":"P2 Extension: polyproto-chat","text":"
TODO
TODO: This is a work in progress. Chat-related content is currently being migrated over from the polyproto-core specification. This document is not at all complete.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#4-federating-directgroup-messages","title":"4. Federating direct/group messages","text":""},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#41-direct-messages","title":"4.1 Direct messages","text":"
Federating direct messages is straightforward. When Alice sends a message to Bob, their client will send the message to Bob's home server via an API request. Bob's home server will then send the message to Bob's client via an established WebSocket connection, and vice versa.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#42-group-messages","title":"4.2 Group messages","text":"
Group messages work just like guilds, in the sense that data is stored by the home server of the group's creator, meaning that all group members will have to communicate with the group creator's home server. If the group creator leaves the group, the ownership of the group is transferred to another member. The group chat stays on the group creator's home server.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#6-encrypted-channels-and-groups","title":"6. Encrypted channels and groups","text":"
Note, that in the below sequence diagrams, the MLS Welcome message and the MLS Group notify message are all encrypted using the identity key of the recipient.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#61-encrypted-guild-channels","title":"6.1 Encrypted guild channels","text":"
Encrypting a guild channel is done by a client with the MANAGE_CHANNEL permission. Upon successfully requesting enabling encryption of a channel, all future messages in it will be encrypted. Joining an encrypted channel is done by sending a join request to the server. The server will then notify the channels' members of the join request. The members will then decide whether to accept or reject the join request. If the join request is accepted by any member, that member will initiate the MLS welcoming process. If the member finds that the join request is invalid (perhaps due to an invalid KeyPackage), the join request must be denied. It is imperative that join requests are verified correctly by the server.
Text Only
Charlie Server Alice Bob\n | | | |\n | Channel join request + KeyPackage | | |\n |--------------------------------------------->| | |\n | | | |\n | | Notify group of join request | |\n | |----------------------------------- | |\n | | | | |\n | |<---------------------------------- | |\n | | | |\n | | Channel join request + Charlie's KeyPackage | |\n | |------------------------------------------------>| |\n | | | |\n | | | Verify Charlie's KeyPackage |\n | | |------------------------ |\n | | | | |\n | | |<----------------------- |\n | | | |\n | | Notify group of new member: Charlie | |\n | |<------------------------------------------------| |\n | | | |\n | | Encrypted MLS Welcome | |\n | |<------------------------------------------------| |\n | | | |\n | | Forward: Notify group of new member: Charlie | |\n | |------------------------------------------------------------------------------>|\n | | | |\n | Forward: Notify group of new member: Charlie | | |\n |<---------------------------------------------| | |\n | | | |\n | Forward: encrypted MLS Welcome | | |\n |<---------------------------------------------| | |\n | | | |\n
Fig. 3: Sequence diagram of a successful encrypted channel join in which Alice acts as a gatekeeper. The sequence diagram assumes that Alice can verify Charlies' public key to indeed belong to Charlie, and that Alice accepts the join request."},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#62-encrypted-direct-messages","title":"6.2 Encrypted direct messages","text":"
Adding another person to a direct message is not possible, and would not make much sense, as the new person cannot see any messages that were sent before they joined the group. If Alice wants to add Charlie to a direct message with Bob, she will have to create a new direct message with Bob and Charlie.
Text Only
Alice Server Bob\n| | |\n| Request Bob's KeyPackages | |\n|--------------------------------------------->| |\n| | |\n| Bob's KeyPackages | |\n|<---------------------------------------------| |\n| | |\n| Verify Bob's KeyPackages | |\n| ----------------------- | |\n| | | |\n|<----------------------- | |\n| | |\n| Notify group of new member: Bob | |\n|--------------------------------------------->| |\n| | |\n| Encrypted MLS Welcome | |\n|--------------------------------------------->| |\n| | |\n| | Forward: New group member: Bob |\n| |--------------------------------->|\n| | |\n| | Forward encrypted MLS Welcome |\n| |--------------------------------->|\n| | |\n
Fig. 4: Sequence diagram of a successful encrypted direct message creation.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#63-encrypted-group-messages","title":"6.3 Encrypted group messages","text":"
Encrypted group messages work by using the traditional MLS protocol, with the additional concept of group owners. Only group owners can add new members to the group and forcibly remove others from the group. The Group owner is determined by the Client-Server API.
Text Only
Alice (gatekeeper) Server Bob Charlie\n| | | |\n| Request Bob's KeyPackages | | |\n|------------------------------------------------->| | |\n| | | |\n| Bob's KeyPackages | | |\n|<-------------------------------------------------| | |\n| | | |\n| Verify Bob's KeyPackages | | |\n|------------------------ | | |\n| | | | |\n|<----------------------- | | |\n| | | |\n| Notify group of new member: Bob | | |\n|------------------------------------------------->| | |\n| | | |\n| Encrypted MLS Welcome | | |\n|------------------------------------------------->| | |\n| | | |\n| | Forward: New group member: Bob | |\n| |-------------------------------------->| |\n| | | |\n| | Forward encrypted MLS Welcome | |\n| |-------------------------------------->| |\n| | | |\n| Request Charlie's KeyPackages | | |\n|------------------------------------------------->| | |\n| | | |\n| Charlie's KeyPackages | | |\n|<-------------------------------------------------| | |\n| | | |\n| Verify Charlie's KeyPackages | | |\n|---------------------------- | | |\n| | | | |\n|<--------------------------- | | |\n| | | |\n| Notify group of new member: Charlie | | |\n|------------------------------------------------->| | |\n| | | |\n| Encrypted MLS Welcome | | |\n|------------------------------------------------->| | |\n| | | |\n| | Forward: New group member: Charlie | |\n| |-------------------------------------->| |\n| | | |\n| | Forward: New group member: Charlie | |\n| |------------------------------------------------>|\n| | | |\n| | Forward encrypted MLS Welcome | |\n| |------------------------------------------------>|\n| | | |\n
Fig. 5: Sequence diagram of a successful encrypted group creation with 3 members.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#64-joining-new-devices-from-existing-users","title":"6.4 Joining new devices from existing users","text":"
Regardless of channel or group permissions, a user join request from a new device should be accepted by default.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#65-best-practices","title":"6.5 Best practices","text":"
- In case of encrypted guild channel join requests, it may be a good idea to treat multiple join requests from the same user with different clients as a single join request, when it comes to UI/UX.
- Joining an encrypted channel, even from an already established member with a new device, should be an event distinctly visible to all members of the channel. This is to prevent a malicious user from joining a channel without the other members noticing.
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/","title":"P2 Extension: polyproto-mls","text":"
TODO
This is a work in progress. MLS-related content is currently being migrated over from the polyproto-core specification. This document is not yet complete.
v0.1.0-alpha.1 - Treat this as an unfinished draft. Semantic versioning v2.0.0 is used to version this specification. The version number specified here also applies to the API documentation.
- P2 Extension: polyproto-mls
- 1. Encryption
- 1.1. KeyPackages
- 1.1.1 Last resort KeyPackages
- 1.2 Initial authentication
- 1.3 Multi-device support
The following sections describe the additional behavior that polyproto implementations must implement to support encryption via the Messaging Layer Security (MLS) protocol.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#1-encryption","title":"1. Encryption","text":"
About MLS
Polyproto offers end-to-end encryption for messages via Message Layer Security (MLS). polyproto compliant servers take on the role of both an Authentication Service and a Delivery Service in the context of MLS.
MLS is a cryptographic protocol that provides confidentiality, integrity, and authenticity guarantees for group messaging applications. It builds on top of the Double Ratchet Algorithm and X3DH to provide these security guarantees.
Implementations of polyproto can opt to support encryption to secure communication channels. The selected security protocol for all polyproto implementations is the Messaging Layer Security protocol, given its feasibility within the implementation context. MLS inherently supports negotiation of protocol versions, cipher suites, extensions, credential types, and extra proposal types. For two implementations of polyproto to be compatible with each other in the context of encryption, they must have overlapping capabilities in these areas.
The following sections explain the additional behavior that polyproto implementations utilizing MLS must implement.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#11-keypackages","title":"1.1. KeyPackages","text":"
Warning
The sections 1.1 and 1.1.1 are not exhaustive and do not cover all aspects of MLS and KeyPackages. They exist solely to give a general overview of how KeyPackages are used in polyproto. Please read and understand the MLS specification (RFC9420) to implement polyproto correctly.
A polyproto compliant server must store KeyPackages for all clients registered on it. The KeyPackage is a JSON object that contains the following information:
JSON
{\n \"protocol_version\": \"<Version>\",\n \"cipher_suite\": \"<CipherSuite>\",\n \"init_key\": \"<HPKEPublicKey>\",\n \"leaf_node\": \"<LeafNode>\",\n \"extensions\": \"<Extensions>\",\n}\n
protocol_version denotes the MLS protocol version.cipher_suite indicates the used cipher suite for this KeyPackage. Note that a server can store many KeyPackages for a single actor, to support various cipher suites.init_key is a public key for encrypting initial group secrets.leaf_node is a signed LeafNodeTBS struct as defined in section 7.2. Leaf Node Contents in RFC9420. A LeafNode has information representing a users' identity, in the form of the users' ID-Cert for a given session or client. The LeafNodeTBS is signed by using the actor's private identity key.extensions can be used to add additional information to the protocol, as defined in section 13. Extensibility in RFC9420.
A KeyPackage is supposed to be used only once. Servers must ensure the following things:
- That any KeyPackage is not given out to clients more than once.
- That the
init_key values of all KeyPackages are unique, as the init_key is what makes the KeyPackage one-time use. - That the contents of the
LeafNode and the init_key were signed by the actor who submitted the KeyPackage.
Because KeyPackages are supposed to be used only once, servers should retain multiple valid KeyPackages for each actor, alerting clients when their stock is running low. Consult the \"Registration needed\"-API for more information about how servers should request new KeyPackages from clients. Servers should delete KeyPackages when their validity lapses.
Servers only store KeyPackages for home server users, not for foreign users.
About keys
It is recommended that keys are generated using the EdDSA signature scheme, however, other signature schemes may be used as well. Consider, that intercompatibility can only be guaranteed if all communicating parties have an overlapping set of supported signature schemes.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#111-last-resort-keypackages","title":"1.1.1 Last resort KeyPackages","text":"
A \"last resort\" KeyPackage, which, contrasting regular KeyPackages, is reusable, is issued when a server runs out of regular KeyPackages for an actor. This is to prevent DoS attacks, where malicious clients deplete all KeyPackages for a given actor, blocking that actor's inclusion into encrypted groups or guild channels.
Servers are to replace a \"last resort\" KeyPackage after it has been used at least once by requesting one from the client.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#12-initial-authentication","title":"1.2 Initial authentication","text":"
During the initial authentication process, a client must provide at least one KeyPackage and one \"last resort\" KeyPackage to the server, in addition to the required registration information.
The public identity key inside the LeafNode of this KeyPackage corresponds to the public identity key found inside a clients' ID-Cert.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#13-multi-device-support","title":"1.3 Multi-device support","text":"
polyproto servers and clients employing encryption must support multi-device use. The MLS protocol assigns each device a unique LeafNode and prohibits key sharing across devices. Each device offers distinct KeyPackages and an own ID-Cert.
TODO: Integrate this from the core spec
Text Only
A server identity key's lifetime might come to an early or unexpected end, perhaps due to some sort\nof leak of the corresponding private key. When this happens, the server should generate a new\nidentity key pair and broadcast the\n[`SERVER_KEY_CHANGE`](/docs/APIs/Core/WebSockets/gateway_events.md#server_key_change) and\n[`LOW_KEY_PACKAGES`](/docs/APIs/Core/WebSockets/gateway_events.md#low_key_packages) gateway events\nto all clients. Clients must request new ID-Certs (through a CSR), and respond appropriately to the\n[`LOW_KEY_PACKAGES`](/docs/APIs/Core/WebSockets/gateway_events.md#low_key_packages)\nevent. Should a client be offline at the time of the key change, it must be informed of the change\nupon reconnection.\n\n!!! note\n\n A `LOW_KEY_PACKAGES` event is only sent by servers which use MLS encryption. Server/Clients not\n implementing MLS encryption can safely ignore this event.\n
"},{"location":"Type%20Definitions/","title":"meow","text":""},{"location":"Type%20Definitions/auth/","title":"more meow :3","text":""},{"location":"Type%20Definitions/core/","title":"more meow :3","text":""},{"location":"blog/","title":"Blog","text":""},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/","title":"Self-updating structs, moving blog posts to GitHub, and more!","text":"
Introducing self-updating structs, explaining how they work, and what they are good for. Also, moving blog posts to GitHub, and other improvements.
It has been a while since the last update post - 1 month to be precise! I haven't gotten around to writing one of these, mostly because of personal time- and energy constraints. However, now that these resources are finally replenishing again, I figured that it is once again time!
"},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/#moving-blog-posts-to-github","title":"Moving Blog Posts to GitHub","text":"
This is a pretty self-explanatory point. I thought, that opencollective would find more use by me and other polyphony-curious folk, however, this didn't go as planned. Also, opencollective made their Discord embeds really poopy, which is why I am moving all the blog posts over to GitHub.
"},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/#a-big-one-self-updating-structs","title":"A big one: Self-updating structs","text":"
Ideally, you want entities like Channels, Guilds, or Users to react to Gateway events. A Gateway event is basically a message from Spacebar/Discord to you, which says: \"Hey, User x has changed their name to y!\". If you can reflect those changes immediately within your code, you save yourself from having to make a lot of requests and potentially getting rate-limited.
This is exactly what Self-updating structs set out to solve. The first implementation was done by @SpecificProtagonist and me (thank you a lot again, btw) on the 21st of July. However: This implementation, being in its' infancy, has had some design flaws, which to me made pretty clear, that this whole thing needed to be thought through a little better.
The second iteration of these Self-updating structs was finished... today, actually, by me. It saves memory compared to the first iteration by storing unique objects only once, instead of n = how many times they are being referenced-times. While this way of doing things is really efficient, it also has been a pain in the ass to make, which is precisely the reason why this took me so long. I've learned a lot along the way though.
The public API has also gotten a lot better in \"v2\". This is mostly because I am a big believer in writing tests for your code, and through writing what are essentialy real-world-simulation-examples, I noticed how repetitive or stupid some things were, and thus could improve upon them.
Having this whole thing finished is a big relief. This self-updating thing is an essential feature for any Discord/Spacebar compatible library, and I think that we implemented it very nicely.
"},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/#documentation-and-other-improvements","title":"Documentation and other improvements","text":"
@kozabrada123 took it upon himself to re-write a lot of the codes' Documentation. Thanks for that! This will massively improve the ease of use of this library - both when developing for and with it. koza also improved our CI/CT pipeline by incorporating build-caching into it, which speeds up builds.
This has been the last month of Polyphony. In the coming weeks, I will be working on - Implementing self-updating-struct behavior for every struct which needs it - Fixing bugs - Adding more features, like emojis, 2FA, Guild Settings, etc.!
See ya next time!
"},{"location":"blog/2023/08/29/chorus-alpha-010/","title":"chorus Alpha 0.1.0","text":"
We are alpha now! As of 2 days ago, the first Alpha of Chorus, Version 0.1.0, has been released for everyone to look at and use on crates.io!
So, is the library complete now? No. And yes! It's, well, complicated... Let me explain!
Chorus is at a point where I can comfortably say that, if you take voice-support out of the calculation for a bit, the foundation feels rock-solid, easy to work with and easily expandable. However, to stay with our house/building metaphor for a bit, the walls aren't painted yet, there's barely any furniture and not all of the electrical outlets have been installed yet.
Okay, enough with this bad metaphor; What I meant to convey is, that a lot of the API endpoints have not yet been implemented, and there are at least a few points we haven't addressed yet - like Gateway Error Handling, to name an example.
But for an early Alpha, this, in my opinion, is absolutely acceptable. Implementing API endpoints is something that probably someone who is entirely new to Rust could do, given that we've streamlined the procedure so much, and the other stuff can comfortably be fixed without having to do any major changes to the internals.
I, for one, am currently experimenting around with the Polyphony Client, which, by the way, will likely be written with Iced as a GUI Framework, not GTK. I have no prior experience in GUI/Desktop Application development, but I am feeling more confident than ever and I'm eager to learn all there is to know about these topics.
That's that! Seeya next time. Cheers,
Flori
"},{"location":"blog/2023/09/02/getting-started-with-the-polyphony-client/","title":"Getting started with the Polyphony Client","text":"
Us labeling Chorus to be in a public-alpha state was really great news for me, for a lot of reasons! It marked a point in Polyphonys history where, after all these months of work, we agreed upon the fact that what we have is good enough to be shown to the public, and that's always a nice thing when investing so much of your free-time into a project. The other main reason why this is such a great thing is, because this alpha state (at least to me) means, that the public API is kind-of stable, or at least stable enough so that I, the project lead, can rely upon the fact that all the public methods will not, in fact, be replaced in 4 days.
This means, that I can finally start working on the Client! And I have done that! For the past 2? 3? Days, I've been tinkering around with Iced-rs (a really, really great UI framework for Rust, written in Rust) and the client repository to create the 'skeleton' of the application. While this is definitely not trivial, especially since I have no prior experience in desktop application development, it's also not too hard either.
While Iced is not mature yet, and \"how-to\" guides, as well as the promised Iced-book, are still largely missing, the maintainers have done a great job with providing a LOT of code examples and solid rustdocs. It's a fun library/framework to work with, and the Elm-inspired approach of dividing up State, Messages, View- and Update-Logic feels really intuitive and seems to make sure that your Application will never end up in an unexpected state.
That's all I have for today. Thanks for reading this! Here's a video of multi-user login already working ^^
"},{"location":"blog/2023/11/23/porting-chorus-to-webassembly--client-update/","title":"Porting chorus to WebAssembly + Client Update","text":"
What the current state of GUI libraries in Rust means for Polyphony and chorus, and why we are porting chorus to WebAssembly.
Hi all!
To make this part of the post short: The web-based client will be worked on before the native one, if there even ever will be one. The reason is that no currently available native Rust GUI library meets the standards I'd like to see when using it to build an application I am putting my name behind. I'd like to have - accessibility - great styling - cross compilation - memory safety
and the current state of Rust GUIs essentially tells me to \"pick three\", which is unacceptable to me. A WebAssembly based application is the best we'll get for now, and I am fine with that.
Compiling to WebAssembly isn't all that easy though: The wasm32-unknown-unknown target intentionally makes no assumptions about the environment it is deployed in, and therefore does not provide things like a net or filesystem implementation (amongst other things). Luckily, adding support for this compilation target only took me a full 40h work week [:)], and we are now the first Rust Discord-API library (that I know of) to support this target.
You might not have yet heard much about WebAssembly: In the past, web developers could only really use three languages - HTML, CSS, and JavaScript - to write code that browsers could understand directly. With WebAssembly, developers can write code in many other languages, then use WASM to convert it into a form the browser can run.
This is particularly helpful for programs that require a lot of computing power, like video games or design software. Before, running such programs in a browser would be slow or impossible. WebAssembly can make these run smoothly, right in your web browser.
Overall, WebAssembly is expanding the kinds of applications that can be run on the web, making the web a more flexible and powerful place to work and play. Compiling Chorus for WASM allows us to leverage this fairly new technology and bring all of Rusts benefits into a web context.
The next blog post will likely be about progress with the web-based client. See ya until then! :)
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/","title":"Account migration in polyproto","text":"
Account migration is an important and difficult thing to get right in federated systems. In this blog post, I will outline how I imagine account migration to work in polyproto, and what benefits this approach brings.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#account-migration-in-polyproto","title":"Account migration in polyproto","text":"
It seems that striking a good balance between user experience, convenience and privacy has been a difficult task for many federated systems, when it comes to account migration. polyprotos' approach to how data is distributed and stored, and how identities are managed, makes it possible to have a very smooth and secure account migration process.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#the-problem","title":"The problem","text":"
Using Mastodon as an example; When a user wants to move from one instance to another, they have to create a new account on the new instance, and follow all the people they were following on the old account. All the toots and other data from the old account are left behind, and you do not have a way of porting them over to the new account. This is a problem that has been around for a long time, and it is not just a problem with Mastodon, but with many other federated systems as well.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#how-polyproto-works-briefly","title":"How polyproto works, briefly","text":"
In polyproto, your federation ID, e.g. xenia@example.com, is what identifies you. If you want to use this identity on a client, your client will generate a key pair for a certificate signing request, and send this request to your home server. Given that you didn't provide any invalid data, your home server will sign the certificate, and send it back to you.
Any data you send to anyone - be it a chat message, a social media post, or anything else - is signed using your private key. This signature can be verified by anyone using your public key, which is part of the certificate you received from your home server. To check a certificates' validity, you can ask the home server for its root certificate, and verify the signature on the certificate you received.
This means:
- All the data you send is cryptographically tied to your identity
- Anyone can verify that the data was actually sent by you
- Anyone can verify that the data was not tampered with by anyone else
- Everybody can verify that you are who you say you are
This is even true when you are sending data to a different server than your home server.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#migrating-an-account-on-polyproto","title":"Migrating an account on polyproto","text":""},{"location":"blog/2024/02/07/account-migration-in-polyproto/#low-data-centralization","title":"Low data centralization","text":"
Fundamentally, the process of migrating an account in polyproto relies mostly on changing data ownership, rather than moving data around. This works best in scenarios where data is highly distributed, and not stored in a central location.
Example
This might be the case in a social chat messaging system similar to Discord, where messages are stored on the servers of the people hosting the chat rooms.
When you want to move your account from one server to another, you:
- First, create a new account on the new server
- Then, you configure the new account to back-reference the old account
- Next, if you are able to, you tell your old home server about the move
- Last but not least, you verify to the servers storing your data that you are the same person as the one who created the old account. The servers then update the data ownership to your new account. This is done by using your old private key(s), in a way that does not reveal your private key(s) to anyone else.
If applicable, your friends and followers will also be notified about the move, keeping existing relationships intact.
Note
This entire process does not rely on the old server being online. This means that the process can be completed even if the old server is down, or if the old server is not cooperating with the user.
However, including the homeserver in the process adds to the general user experience. If you, for example, have included your federation ID as part of another, non-polyproto social media profile, the old server can automatically refer people to the new account.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#moving-data","title":"Moving data","text":"
Should data actually need to be moved, for example when the old server is going to be shut down, or if the centralization of data is higher, the migration process is extended by a few steps:
- Using the old account, your client requests a data export from your old home server.
- The old home server sends you a data export. Your client will check the signatures on the exported data, to make sure that the data was not tampered with.
- You then import the data into your new account on the new home server.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#conclusion","title":"Conclusion","text":"
polyproto's approach to account migration is very user-friendly, and does not require the user to do anything that is not already part of the normal usage of the system. The process is also very secure, as it relies on the cryptographic properties of X.509 certificates, and also works across a highly distributed data model, which, in my opinion, is how the internet should be.
The biggest drawback to this approach is that there are a whole lot of web requests involved. Depending on the amount of data, this can take some minutes or possibly even hours.
It is also worth noting that all of this does not require any new or young technology. polyproto relies on X.509 certificates, which have been around for a long time, and are widely used in many different applications. This means that the technology is well understood, and that there are already many great tools in all sorts of programming languages available to work with it. From my point of view, there is no need to reinvent the wheel.
I hope that this article has given you a good understanding of how account migration works in polyproto. If you have any questions or feedback, feel free to reach out to me via E-Mail, where I can be reached under flori@polyphony.chat. OpenPGP is supported, and my public key can be found on keys.openpgp.org (click to download pubkey)
"},{"location":"blog/2024/02/19/x509-in-polyproto/","title":"Certificates, please: X.509 in polyproto","text":"
This blog post covers a bit about how and why X.509 is used in polyproto, and how we try to make the process of implementing your own server and incorporating it into an existing network a little easier.
Authors' note
Before knowing and reading about the X.500- and PKCS-series of RFCs, I legitimately thought, that implementing an own certificate standard for polyproto would be a good idea! Looking back, this is incredibly naive. But learning new things and improving myself is one of the biggest joys I experience when writing software, so this humbling experience was totally worth it for me, personally.
polyproto is a federation protocol that uses X.509 Public Key Infrastructure (PKI) to prove and federate your identity across a whole network of decentralized services.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#x509","title":"X.509","text":"
Specifically, polyproto leverages the already well-documented and widely used X.509 standard at its core. X.509 was chosen over OpenPGP because of its comparative simplicity. The Web of Trust from OpenPGP often requires active user input to assign trust levels to users and their keys, which is not inline with our ideas and goals for user experience in a decentralized system. Ideally, decentralization and federation is as seamless as possible for the end-user, and X.509 with its Certificate Authority (CA for short) model is the better fit for such a goal. In fact, X.509 can be so seamless to the end-user, that you have probably forgotten that you are already using it right now!
HTTPS (SSL/TLS) certificates are likely the most popular form of digital certificate out there, and they\u2019re implemented in a way, where the only time us humans ever have to think about them, is when our browser tells us that a certificate from a website we\u2019re trying to visit, is not valid anymore.
This popularity is great news for polyproto, because it means that mature tooling for all sorts of programming languages exists today, along with tutorials and documentation, teaching potential implementers how everything works.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#how-polyproto-uses-x509-briefly","title":"How polyproto uses X.509, briefly","text":"
In polyproto, home servers act as Certificate Authorities, while each client you connect from has its own end-user Certificate, issued by your home server. With certificates, you can prove your identity to any person or server at any time. Certificates are also used to verify the integrity of data sent across the polyproto network.
If servers and clients have well-implemented cryptography, it should be extremely unlikely - if not impossible - for non-quantum-based, non-supercomputer-cluster home servers to alter the contents of a message before passing them on to the recipient.
Authors note
Technically, polyproto and X.509 absolutely support Post-Quantum Hybrid Digital Signatures. If these Hybrid Digital Signatures use well-made Post-Quantum Signature schemes and are implemented well, polyproto also offers post-quantum-computing resilience. There seems to be very little, easy to understand reading material on hybrid schemes out there. The best/most easy to understand definition or explanation of hybrid schemes I could find is this one, in the document \"A Hybrid Signature Method with Strong Non-Separability\".
In short, clients generate a PKCS #10 Certificate Signing Request (CSR). This CSR includes some information about the client. In polyprotos case, this information is:
- session ID
- federation ID
- algorithm used to generate the public key attached to the CSR
- the public key attached to the CSR
- a signature which is verifiable using the attached public key, validating all of the aforementioned information
This CSR is sent to your home server, which verifies this information and in turn responds with a polyproto X.509 Certificate (ID-Cert).
Home servers get their root certificate by self-signing a CSR. Unlike actor/client certificates, the home server root certificate features X.509 extensions such as the \"Basic Constraints\" attribute, marking its certificate as a CA certificate, allowing the home server to sign CSRs using this certificate.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#but-its-not-all-perfect","title":"But it\u2019s not all perfect.","text":"
Root Certificates in the context of HTTPS and the modern, SSL/TLS protected web are a big source of centralization. This centralization might be necessary to a degree, but it inevitably means less plurality, and way more hoops to jump through, should you also want to be a CA.
To give context for those who might need it, essentially, every certificate for every website out there has to be able to be traced back to one of the root certificates installed on your internet-capable device's operating system or web browser. This creates an incredible amount of centralization, because one Root Certificate Authority is directly responsible for hundreds of thousands, if not millions of websites. This dependency on a few privileged Root CAs has been monetized, which is why getting an SSL/TLS certificate for your website used to cost you money (and depending on who you are, it might still be that way). Nowadays though, Let's Encrypt exists, offering free SSL/TLS certificates, with the caveat that these certificates are only valid for three months at a time.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#what-can-we-do-about-this","title":"What can we do about this?","text":"
To try and keep open polyproto networks to stay open for everyone, polyproto should make centralization to the degree of modern-day SSL/TLS at infeasible.
An approach we are taking is limiting the length of the certification path.
In X.509, to validate and trust a certificate, you must also trust all the other certificates leading up to the Root Certificate of the Certificate Tree.
graph LR\n A[Root CA] --> B[CA 1]\n A --> C[CA 2]\n B --> D[Middleman]\n D --> E([Leaf Certificate 1])\n C --> F([Leaf Certificate 2])
Example
To trust Leaf Certificate 1, one would have to also trust the certificates held by the Middleman CA, CA 1 and the Root CA.
This path from the certificate you are actually trying to validate to the Root Certificate is referred to as the certification path. By arbitrarily limiting the length of this path, it becomes harder for one certificate authority to issue and manage a great (1.000.000+) number of certificates, due to the increasing amount of processing power required to handle web requests and to verify and sign CSRs.
In polyproto, the maximum length of this certification path is 1, meaning a Root Certificate may only issue leaf certificates. Cutting out middlemen makes it hard to scale to monstrous levels of centralization, as the control one CA can have over the entire network is limited.
All of these factors combined should always make developing or hosting your own home server a viable option.
Authors note
To clarify, this does not mean that polyproto servers will only be able to handle a small amount of users, or that polyproto is designed for small-userbase scenarios. A well-implemented and fast home server implementation should, with the given resources, be able to handle a great number of registered users. This shallow-depth trust model should aid in stopping trust hierarchies with great amounts of influence over the network from forming.
However, real-life power distribution scenarios can be be unpredictable, which means that the efficacy of limiting the certificate path length as a measure to prevent centralization can only be proven when polyproto is being deployed in the real world.
If you have any questions or feedback, feel free to reach out to me via email, where you can reach me under flori@polyphony.chat. OpenPGP is supported, and my public key can be found on keys.openpgp.org (click to download pubkey)
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/","title":"Work on polyproto and taking a break","text":"
In this little update post I write about what I've done in the last couple of weeks alongside talking about taking just a little break (don't worry, y'all are not getting rid of me!)
It's been more or less two weeks since the last post - time for the next one!
A good amount of commits have been since the X.509 in polyproto was published. Let's break them down a little, shall we?
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#certificate-signing-requests","title":"Certificate Signing Requests","text":"
The polyproto crate can now be used to create very basic - but to the best of my knowledge fully RFC compliant - Certificate Signing Requests! This is cool, because Certificate Signing Requests are how all Actors (Users) in polyproto will request a Certificate from their home server. The generated CSRs can be fully verified using the OpenSSL/LibreSSL CLIs, which is very important, as these two applications are the industry standard when it comes to working with cryptographic standards like X.509.
Specifically, polyproto uses the well-defined PKCS #10 standard to pack up and transport all the needed CSR information to your future home server.
The next steps here are:
- Creating validators for the information supplied in the CSRs
- Implementing methods to create an ID-Cert from a CSR
- Write great documentation for what exactly the data inside of the ID-CSR has to look like to be valid
...and as you might have already guessed, I am already working on all of these things! :) They just take time
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#cleaning-up","title":"Cleaning up","text":"
As fun as designing APIs and software architecture is for me, I don't yet always get all of it right on the first try. This is fine though, as long as you recognize the mistakes you've made, learn from them and clean the mess you've made.
I noticed that, as well-meant as some of the traits and trait bounds I've added, they made implementing polyprotos' base types and traits a lot harder than needed. I've been chipping away at the unnecessary and redundant bits, removing some of these traits entirely.
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#updating-the-specification-document","title":"Updating the specification document","text":"
I really wanted to get started on a reference polyproto implementation before finishing the specification document. This might seem a little counter intuitive, but my thought process was, that implementing the crate in code would force me to think about everything from scratch again, which would make it much easier to spot mistakes I potentially made when writing the specification documentation. These mistakes would primarily be:
- Information that is there, but unimportant
- Information that is important, but not there
- Information that is important, there, but wrong
This turned out to be right. I have added a lot of \"TODO\"s and \"FIXME\"s into the specification document since started working on the polyproto crate. All of these TODOs have since been worked on and removed! This doesn't mean that the specification document is now perfect, but it's already better than before, and it'll only get better as I continue to work on the crate!
Another, notable thing that happened is removing the auth-part from the core polyproto protocol! You might be thinking \"whaaaat? does that mean that there will be no authentication in polyproto??\" but I can assure you, that that's not what this means. Removing the authentication endpoints from the core protocol means that polyproto extensions can now choose authentication technologies and methods for themselves, instead of being forced to implement a bunch of REST-based authentication endpoints they might not even want or use anyways.
I would like to thank @laxla@tech.lgbt for this idea! :> Collaboration and feedback are truly great things, and I am happy to have such a nice group of people on Discord and Matrix who are genuinely interested in the silly thing I/we want to do with Polyphony and polyproto :)
Now for the perhaps biggest and probably most important announcement:
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#taking-a-little-break-for-my-silly-mental-health","title":"Taking a little break for my silly mental health","text":"
It just dawned on me that March 8th marks the one year anniversary of Polyphony!! That's genuinely so cool, and means that this is the project I have worked on the longest for, out of all of my personal projects.
So yeah - it's been almost a year now! And not a lazy one for me, either.
Content warning
The following paragraph covers the topics of anxiety and depression. If you would not like to read about this, feel free to scroll down until you see a big green box with a check mark. The box indicates that it is safe for you to read again!
Big shocker: I am \ud83d\udc7b\ud83d\udc7b\ud83d\udc7b\ud83d\udc7b depreeeeeeessed \ud83d\udc7b\ud83d\udc7b\ud83d\udc7b\ud83d\udc7b\ud83d\udc7b, and have been for the past... 4-6 years of my life. In that time, I have experienced the absolute lowest points of my life. Luckily, I have the absolute privilege to have a great therapist who I have been with for 2 years now, and I am also on medication which already does a good job (most of the time) at taking the edge off the depression.
As it has been explained to me by my therapist, medication should only be a crutch, though. It should not be the tool you should solely rely on for the rest of your life to deal with extreme (social) anxiety and depression. Other, non-medication-related options should be tried, to potentially get you to stop having to take medication to feel non-completely-absolutely-positively-awful every day.
One of these options is therapy, and, as I've mentioned, I've already been doing that for 2+ years now. It has helped me a great, great deal already, and I can absolutely encourage anyone reading who is feeling similarly to how I've described and who is in the lucky position to get (or at least be put on a waiting list for) therapy, to take the first step. It isn't easy; it can actually feel really really scary at first. But do believe me when I say that a good therapist can absolutely help you to get better.
But one hour of therapy a week can sadly only do so much. This is why I, with the encouragement of my friends, loved ones (particularly my lovely, lovely girlfriend) and my therapist, have decided to admit myself into a mental health clinic that specializes in the treatment of depression, anxiety disorders and the like.
Safety checkpoint reached!
It's now over! :)
Starting on March 20th, I will be leaving my everyday life, my girlfriend, my friends, laptop, work, personal projects and everything else behind to go there, and hopefully leave a good bad part of me behind when I come back.
The clinic is far away though, and leaving absolutely everything behind for a month or possibly a little longer is really, really scary to me. However, I think and hope that the metaphorical plunge into icy water will be worth it for me and my mental health.
When I come back, I'll be better than I was before, which will also mean that I can hopefully be more happy and productive in all aspects of my life, including Polyphony.
If you're reading this on or after March 20th, then see you on the other side :) I hope the grass is greener there!
BEGPOSTING ON MAIN
I am lucky and extremely privileged to have been growing up in Germany, a country with a (mostly) functioning social welfare system and universal health care. If this wasn't the case, I'd likely be absolutely unable to afford to put myself into such good care. Germany doesn't pay for everything though, and the train rides to and from the clinic will likely be expensive for me, as will the 10\u20ac daily fee for staying at a clinic (capped at 280\u20ac).
I can currently afford this without financially ruining myself, so don't worry about that. However, this whole endeavour will take a good chunk out of my current savings. Thus, if you'd like to donate to my ko-fi to help me cover the costs, it would mean a lot to me! <3
Please only do so if you are in a stable financial standing yourself, though. As I said, with or without tips, I'll manage. :)
"},{"location":"blog/2024/06/01/polyproto-extensions/","title":"polyproto extensions.","text":"
polyproto is a new federation protocol. Its main focus is enabling seamless participation of one actor on many different servers. The core specification lacks routes for sending any sort of user generated data anywhere, though. What is up with that?
"},{"location":"blog/2024/06/01/polyproto-extensions/#to-federate-is-to-be-familiar","title":"To federate is to be familiar","text":"
If any application wants to participate in the network of polyproto services, it has to speak the same language as those other services. When wanting to send a message to a server that you are authenticated on, your client needs to know exactly what that HTTP request has to look like. This is nothing new. One take on a solution for this problem stems from the people working on the ATProtocol, who created Lexicon. From the atproto website:
Lexicon TL;DR
Lexicon is a global schema system. It uses reverse-DNS names like \"com.example.ping()\". The definitions are JSON documents, similar to JSON-Schema. It's currently used for HTTP endpoints, event streams, and repo records
The core of polyproto is supposed to be infinitely adaptable, to be flexible enough to be used for just about anything, which is why I do not want to force a fixed set of routes onto every single polyproto implementation.
Lexicon sounds interesting and really versatile! However, as mature as the idea itself might be, it is pretty complex and does not yet seem to have good community support in the form of libraries/crates to aid in working with this new schema system. I also do not want to force polyproto integrations to use a (potentially very complex) Lexicon parser and dynamic routing system thingymajig - although having \"no rules\" means, that if you want to build a polyproto service which uses Lexicon, you absolutely can.
"},{"location":"blog/2024/06/01/polyproto-extensions/#we-need-a-common-foundation","title":"We need a common foundation","text":"
I am a big proponent of defining a set of (mutually independent) protocol extensions, which include additionally needed behavior and concrete HTTP routes for building a specific application. This has the following benefits:
- If you'd like to build a polyproto chat client, and there's a polyproto-chat extension, you simply need to add the additional things required by that extension. No need for complex parsing! Code only what you need and do not care about the rest.
- Mutual independence means being able to combine extensions however you'd like. You could, for example, create a chat app with integrated microblogging functionality.
- Developers are free to come up with whatever they want. How about ActivityPub x polyproto? Since polyproto doesn't define a message format, this is absolutely possible!
- Simplicity! polyproto and its \"official\" extensions will always just have plain old REST APIs, for which tooling is readily available. Why bother with something fancy and dynamic, when this does the trick?
On the other hand, everyone now has to agree on one extension to use for a specific application. You cannot participate on servers, which have use an extension which is completely different from the one that your client implements, as an example.
"},{"location":"blog/2024/06/01/polyproto-extensions/#the-polyproto-foundation-get-it-sigh","title":"...the polyproto foundation. Get it? sigh","text":"
To develop, provide and maintain polyproto and some major \"official\" extensions (such as polyproto-chat), creating a non-profit foundation is likely a good idea for a future where polyproto is actually being used in the real world.
This could sort of be seen like the XMPP Standards Foundation which develops and maintains XMPP extensions. Unlike XMPPs extensions however, official polyproto extensions should always be major additions in functionality. As an example: XEP-0084 is the official XMPP extension for User Avatars. An entire 12 point document, which describes one simple feature!
polyproto extensions should either always be a major technological addition, which can be taken advantage of by other extensions (examples for this would be WebSocket Gateways and Messaging Layer Security), or a document describing a set of routes, which define a particular application use case (A Discord-like, a Reddit-like, a Twitter-like, and so on). Having official extensions adhere to these rules ensures that polyproto will not become a cluttered mess of extensions and that it and its extensions are easy to understand and implement, due to less documentation having to be read and written.
"},{"location":"blog/2024/06/01/polyproto-extensions/#is-this-a-bottleneck-for-me-as-a-developer","title":"Is this a bottleneck for me as a developer","text":"
If you are a developer, you might ask yourself:
Question
Implementing common chat behaviour sounds cool in terms of intercompatibility, but doesn't this limit what I can do with my application? I have planned for a cool feature X to exist in my chat service, but that doesn't exist in the protocol extension!
Extensions should be a usable minimum of common behavior that all implementations targeting the same \"class\" of application must share. Implementations can absolutely offer all the additional special/unique features they'd like, though. polyproto clients implementing the same extensions can be treated as clients with a reduced feature set in this case. What is crucial, however, is that the additional features do not prohibit \"reduced feature set clients\" from using the behavior described in the extension, if any sort of federation or interoperability is wanted.
What works
In your implementation of a chat service, users can send each other messages with special effects, such as fireworks, confetti and similar. A different implementation of polyproto-chat is unlikely to see these special effects on their end. However, they can still see the messages' text contents, send replies to the message, and do all sorts of other things as described in this hypothetical polyproto-chat extension.
What doesn't work
In your implementation of a chat service, users can send each other messages with special effects, such as fireworks, confetti and similar. Your implementation requires every client to send information about the special effect they'd like to send with a message - otherwise sending the message fails. If this is the case and you haven't implemented a sort of \"adapter\" for other polyproto-chat clients, these clients will not be able to send any messages to servers running your chat software. This conflicts with the behaviour required by the polyproto-chat extension and is therefore unacceptable.
Also keep in mind that through clever engineering, it might be possible to write adapters for behavior, which should be required in your implementation and conflicts with the base extension. Picking up the \"What doesn't work\" example again, the implementer could simply \"translate\" message sending requests made to the polyproto-chat endpoints and add the required \"special effects\" information, stating that messages sent through polyproto-chat endpoints have no special effects added to them.
"},{"location":"blog/2024/06/01/polyproto-extensions/#closing-words","title":"Closing words","text":"
I am of the opinion that, while this way of having extensions might not be the most technologically advanced solution, it certainly offers many possibilities while being easy to understand and implement.
These are my current plans, ideas and thoughts for making a v1 of polyproto extensible. If you have any thoughts on this matter, please do let me know! You can contact me via email or by writing a message on our Discord.
Thank you for reading! :>
"},{"location":"blog/2024/06/01/polyproto-extensions/#happy-pride-month","title":"Happy pride month! \ud83c\udff3\ufe0f\u200d\ud83c\udf08\ud83c\udff3\ufe0f\u200d\u26a7\ufe0f\ud83d\udc9b\ud83e\udd0d\ud83d\udc9c\ud83d\udda4","text":""},{"location":"blog/2024/10/14/nlnet-grant-application/","title":"NLnet grant application","text":"
The NLnet foundation is a non-profit organization that supports open-source projects. They have a grant program that funds projects that align with their goals. On behalf of Polyphony and the polyproto project, I have submitted an application for a grant of 10,000\u20ac from the NLnet foundation in their funding round of October 2024.
Should we be successful in our application, the grant will be used to fund the development of the Polyphony and polyproto projects, which would rapidly increase the development velocity of both projects and bring them one big step closer to being ready for a public alpha release.
The application required a bunch of different, interesting questions to be answered. I would like to share the answers with you, as they give a good overview of what we are working on, what we are planning to do, and considerations we have made in the past.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#can-you-explain-the-whole-project-and-its-expected-outcomes","title":"Can you explain the whole project and its expected outcome(s)?","text":"
polyproto is a new federation protocol with the goal of offering account portability and an effortless experience for users and developers alike. It is part of the Polyphony Project, which aims to be a truly competitive alternative to centralized, proprietary chat services like Discord. We came up with the idea for polyproto because of a lingering frustration with current federation protocols and their sub-optimal suitability for such a project.
polyproto is not limited to an application in a chat service, and that it is not incompatible with federation protocols such as ActivityPub. It would technically be possible to write a polyproto + ActivityPub server and client to offer new possibilities to the currently existing Fediverse. We want to empower users, not split userbases further.
Our goal is to deliver the Polyphony chat service with polyproto at its core, build great SDKs for other developers to work with, and to also directly work together with other developers to get alternative implementations of polyproto-based chat services for users to choose from. Polyphony should be the ideal, federated and decentralized Discord replacement; a service, that can be used by teenagers, the elderly and anyone in between and which ideally does not require any additional technical knowledge or proficiency to use.
Documentation/Protocol specification: https://docs.polyphony.chat/Protocol%20Specifications/core/ Simplified overview of the protocol (sadly a little dated, but it can give an overview of the basics nonetheless): https://docs.polyphony.chat/Overviews/core/ API Documentation: https://docs.polyphony.chat/APIs/core/ \"polyproto-rs\" Rust crate: https://github.com/polyphony-chat/polyproto-rs Polyphony organization overview: https://github.com/polyphony-chat \"chorus\" API Wrapper for Discord, Spacebar-Chat (formerly \"Fosscord\") and our own server: https://github.com/polyphony-chat/chorus
"},{"location":"blog/2024/10/14/nlnet-grant-application/#have-you-been-involved-with-projects-or-organisations-relevant-to-this-project-before-and-if-so-can-you-tell-us-a-bit-about-your-contributions","title":"Have you been involved with projects or organisations relevant to this project before? And if so, can you tell us a bit about your contributions?","text":"
I (Flori Weber, bitfl0wer) have been following the Spacebar-Chat (formerly \"Fosscord\") project for some time before deciding to start the Polyphony-Chat GitHub organization in March 2023. The contributions I have made to the Spacebar project were limited to additions and overhauls of the projects documentation, because of a lack of TypeScript knowledge, which is the programming language primarily used in the Spacebar organization. Of course, I have prior experience in software development and software design through my work at Deutsche Telekom MMS GmbH, but this is my first project with such a topic. I am part of a software development community called \"Commune\", which is a home for individuals and groups who also have interest in federated and/or decentralized social media, as well as \"reclaiming\" the web from the hands of large corporations. There, I have access to like-minded individuals who are also very much interested in polyproto and in seeing polyproto succeed.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#requested-amount","title":"Requested Amount","text":"
$10.000
"},{"location":"blog/2024/10/14/nlnet-grant-application/#explain-what-the-requested-budget-will-be-used-for-does-the-project-have-other-funding-sources-both-past-and-present","title":"Explain what the requested budget will be used for? Does the project have other funding sources, both past and present?","text":"
There are three key things that I (Flori Weber, bitfl0wer) have been wanting to tackle for months now, but have never been able to do, because of a lack of available personal time or expertise. These three things are:
- Writing new material and, if necessary, reworking existing material that
- Describes, in a condensed form, what polyproto is about, targeted towards people who do not have [a lot of] existing knowledge in the topics of federation and decentralized social networking concepts ($250-400)
- Describes, in a condensed form, how polyproto works, targeted towards developers who might be interested in learning more about the inner workings of polyproto, without needing to read the entire protocol specification document ($250-400)
- Starting to build some sort of \"brand\" by
- Commissioning an artist to create a recognizable logo for polyproto ($200-400)
- Commissioning a frontend developer to build a landing page for our project, since non-developers do seem to prefer information hosted outside of GitHub and plain looking documentation pages. This landing page would also host the written material mentioned in 1. ($500-1500)
- Paying (freelance) developers to expedite this projects' journey to completion, where there are the following tasks we could use additional brains on:
- Paying developers to start integrating our polyproto crate into our \"chorus\" client library and \"symfonia\" server (~$2000)
- Stabilizing and extending the symfonia server to host a first, publicly usable instance (~$1500)
- Getting additional help from UI/UX designers and frontend developers to build a client mockup, then having this mockup translated into a client prototype which can be hosted alongside the symfonia server (~$2000)
- \"Overhaul\"/Refactoring tasks which we, as a group of mainly university students working part time jobs in addition, simply did not yet have the time to get to (~$1000-1500)
This would total to $7700 or $9700, depending on whether the lower or higher estimate is used. Additionally, I would like to extend the domains polyphony.chat and polyproto.org for some years using the funding and top up our prepaid E-Mail server hosted at https://uberspace.de/en/. The domain and the E-Mail server make up most of our current operating costs, ranging between 7-12$ a month. This might not sound like a lot in the grand scheme of things, but I am currently paying for these out of my own pocket as an undergraduate with little income, so being able to potentially reduce monthly expenses is a nice prospect.
We currently have and have never had additional/other sources of funding. Receiving funding from NLNet would thus be our first source of funding.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#compare-your-own-project-with-existing-or-historical-efforts","title":"Compare your own project with existing or historical efforts.","text":""},{"location":"blog/2024/10/14/nlnet-grant-application/#spacebar-chat","title":"Spacebar Chat","text":"
Already previously mentioned, Spacebar Chat is also working on an open-source Discord replacement in form of offering an API-compatible server. However, they do not seem interested in making Spacebar Chat a federated chat application with good user experience, as their primary focus is to reverse-engineer and re-implement the Discord API spec-for-spec.
From talking to Spacebar Maintainers, their code reportedly seems to have accrued noticeable amounts of technical debt which made it undesirable for most of the maintainers to continue development on the server. Being friends with some of the now mostly inactive maintainers, I have considered forking the server repository to have an already mostly working starting ground to work with. However, due to the reports of technical debt and our organizations' unfamiliarity with JavaScript/TypeScript, we have decided to start from scratch, in Rust.
The accumulated knowledge that Spacebar contributors and maintainers have collected in form of documentation has already been of great use for our project, and we are contributing back by updating documentation and creating issues, when we find disparities between the behaviours of our own server implementation and their server implementation.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#xmpp","title":"XMPP","text":"
Much like XMPP, I have decided to make polyproto an extensible protocol.
I am of the opinion that XMPPs biggest downfall is how many extensions there are, and that a server aiming to be compatible with other implementations of XMPP-based chat services should aim to implement all of the XEPs to be a viable choice.
polyproto is actively trying to circumvent this by limiting polyproto extensions (P2 extensions for short) to
-
either be a set of APIs and behaviours, defining a generic(!) version of a service. A \"service\" is, for example, a chat application, a microblogging application or an image blogging application. Service extensions should be the core functionality that is universally needed to make an application function. In the case of a chat application, that might be:
-
Defining message group size granularity: Direct messages, Group messages, Guild-based messages
- Defining what a room looks like
- Defining the APIs and behaviours required to send and receive messages
- Defining the APIs and behaviours required to perform commonly sought after things, such as reacting to a message with an emoji
- etc.
The goal is that all different polyproto-based chat applications should then implement this shared behaviour. Of course, developers may absolutely add their own behaviours and functionality which is perhaps exclusive to their specific implementation. Core functionality remains commonly defined however, which should make all polyproto-based chat applications interoperable in these defined, common behaviours.
- or describe a major technological addition, which can be used in the \"requires\" section of another P2 extension. This \"requires\" section can be thought of like the dependency list of a software package.
Technological additions might be: - Defining APIs and behaviours needed to implement the MLS (Messaging Layer Security) Protocol - Defining APIs and behaviours needed to establish and maintain a WebSocket connection, and how to send/receive messages over this WebSocket connection.
By using clay-brick-sized building blocks instead of more LEGO-sized building blocks like XMPP does, we hope to mitigate this problem that we perceive, while still offering an extensible yet well-defined platform to build on.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#matrixelement","title":"Matrix/Element","text":"
Matrix is perhaps the closest we have yet gotten to federated chat software aimed towards a general audience. However, as a strong believer in user experience - especially how first impressions impact new, non-technical users, I believe that Matrix falls flat in this regard. A lot of peoples first experience with Matrix is the infamous \"Could not decrypt: The senders device has not yet sent us the keys for this message\". The protocol and its sub-protocols are vast and complicated and use bespoke cryptography protocol implementations such as Olm and Megolm, which, in the past, has already been the cause of high-caliber vulnerabilities (see: https://nebuchadnezzar-megolm.github.io/ and, more recently, https://soatok.blog/2024/08/14/security-issues-in-matrixs-olm-library/#addendum-2024-08-14).
Matrix is truly impressive from a technical standpoint. Its extremely low centralized architecture fills a niche which especially people already interested in technology seem to enjoy. However, this invariably results in the fact that user experience has to be compromised. It is of my opinion that while Matrix is relatively good at what it is doing, it is not a good fit to be a potential Discord replacement.
As for a comparison: We are taking a radically different approach to Matrix. Matrix aims for eventually-consistent federation of events using cryptographically fully verifiable directed acyclic event graphs, where as polyproto, and by extension Polyphony, prioritize usability above all, intentionally disregarding highly complex or novel data structures in favor of cryptographic verifiability through digital signatures and simple public key infrastructure.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#what-are-significant-technical-challenges-you-expect-to-solve-during-the-project-if-any","title":"What are significant technical challenges you expect to solve during the project, if any?","text":"
Currently, our trust model acknowledges, that a users home server is able to create sessions and session tokens on the users' behalf, and is thus able to listen in on unencrypted communications, or, in the case of a truly malicious admin, would even be able to send messages on behalf of the user. This is not a novel problem, as it also affects all Mastodon ActivityPub servers in existence. Given that this potential abuse risk has not been a large issue in the Fediverse, we expect this to also not be a major problem. However, I would like to find additional mitigations or even a solution for this problem during further development of polyproto.
Another area that will likely need more work is my current design for how to connect to multiple servers at once: Currently, I expect every client to hold a WebSocket connection with each server that they are communicating with, at once. Depending on the amount of traffic, this could lead to constantly high resource consumption for clients. If this turns out to be the case, I am sure that we can find plenty of software- and protocol-side adjustments and improvements to implement - though it is still a potential technical challenge.
My last major area of concern is how well transmission and de-/serializing of the X.509 based Identity Certificates will work. I am optimistic about this however, since the X.500 series of RFCs are extremely well documented and already deeply explored, so that even if challenges arise in this area, I am certain that there is enough literature on the exact problem we might be facing, and enough people to ask/talk to.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#describe-the-ecosystem-of-the-project-and-how-you-will-engage-with-relevant-actors-and-promote-the-outcomes","title":"Describe the ecosystem of the project, and how you will engage with relevant actors and promote the outcomes?","text":"
As the commercialization of Discord.com steadily increases, it is becoming clear that people are looking for a usable alternative. This is an audience that we are hoping to capture. Our Polyphony Chat service is Discord API compatible, so that actors may use the Polyphony client to interact with both Discord.com and polyproto-chat-based instances, and that existing bots and automations could potentially be ported over very easily. This essentially gives people looking for a Discord replacement exactly what they are looking for, as there should be little to no additional concepts, behaviors or patterns that users have to learn or re-learn to use our service.
As previously touched on, we are blessed to already have made a great amount of connections to like-minded developers also working on similar projects, who are looking optimistically towards polyproto as the tool to use to federate. I also have received explicit permission from Spacebar Maintainers to promote our projects on their Discord Guild, which currently counts 3600 members.
"},{"location":"blog/archive/2024/","title":"October 2024","text":""},{"location":"blog/archive/2023/","title":"November 2023","text":""},{"location":"blog/category/polyproto/","title":"polyproto","text":""},{"location":"blog/category/updates/","title":"updates","text":""},{"location":"blog/category/x509/","title":"X.509","text":""},{"location":"blog/category/chorus/","title":"chorus","text":""},{"location":"blog/category/polyphony/","title":"polyphony","text":""}]}
\ No newline at end of file
+{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Polyphony","text":"
This documentation currently hosts information about polyproto; an advanced, secure and scalable protocol for federation that empowers users.
The core of the protocol lies in the polyproto-core specification and in the federation API routes, which are used to negotiate and establish connections between foreign servers and clients.
"},{"location":"#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/","title":"APIs","text":"
"},{"location":"APIs/MLS/","title":"polyproto-mls","text":"
- Core Routes: Registration needed
- Core Routes: No registration needed
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/","title":"MLS Routes: No registration needed","text":"
TODO
This is a work in progress. MLS-related content is currently being migrated over from the polyproto-core specification. This document is not yet complete. Feel free to contribute by opening a pull request on the docs repository. The API will change (possibly drastically) in the future.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#encryption","title":"Encryption","text":"
Client-Foreign Server API endpoints concerned with encryption related tasks.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#get-keypackages","title":"GET KeyPackage(s)","text":"
/.p2/core/v1/keypackage/:fid
Request KeyPackages - initial encryption keying material - for a specific actor from the server. The requested actor must be registered on this server.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#request","title":"Request","text":""},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#parameters","title":"Parameters","text":"Name Type Description
fid String, Federation ID The Federation ID of the actor whose KeyPackage(s) should be returned."},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#body","title":"Body","text":"
This request has no body.
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#response","title":"Response","text":"200 OK"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#body_1","title":"Body","text":"Type Description JSON-Array of KeyPackage(s), Base64 The actor's KeyPackage(s), Base64 encoded. Each entry in the array corresponds to a different client the requested actor is authenticated on. JSON
[...]\n
"},{"location":"APIs/MLS/Routes%3A%20No%20registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/","title":"MLS Routes: Registration needed","text":"
TODO
This is a work in progress. MLS-related content is currently being migrated over from the polyproto-core specification. This document is not yet complete. Feel free to contribute by opening a pull request on the docs repository. The API will change (possibly drastically) in the future.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#encryption","title":"Encryption","text":"
Client-Home Server API endpoints concerned with encryption, such as KeyPackage management.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#post-add-keypackage","title":"POST Add KeyPackage","text":"
/.p2/core/v1/keypackage/@me
Add a KeyPackage to your KeyPackage store on the server. Only adds KeyPackages to the ID-Cert corresponding to the session token used in the authorization-Header.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#request","title":"Request","text":""},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body","title":"Body","text":"Type Description JSON-Array of KeyPackages One or more KeyPackages to add to the available KeyPackages for this actor. JSON
[ {...}, {...} ]\n
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#response","title":"Response","text":"201 Created"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body_1","title":"Body","text":"
This response has no body.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#put-replace-last-resort-keypackage","title":"PUT Replace Last Resort KeyPackage","text":"
/.p2/core/v1/keypackage_lr
Replace a Last Resort KeyPackage with a new one. Only manipulates Last Resort KeyPackages for the ID-Cert corresponding to the session token used in the authorization-Header.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#request_1","title":"Request","text":""},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body_2","title":"Body","text":"Type Description KeyPackage The KeyPackage to replace the current Last Resort KeyPackage with. JSON
{...}\n
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#response_1","title":"Response","text":"204 No Content"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#body_3","title":"Body","text":"
This response has no body.
"},{"location":"APIs/MLS/Routes%3A%20Registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/auth/","title":"polyproto-auth","text":"
- Authentication Routes: Registration needed
- Authentication Routes: No registration needed
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/","title":"Authentication Routes: No registration needed","text":"
All API endpoints needed for implementing polyproto-auth. This Page only includes routes, for which a client does not need a \"Client-Home Server relationship\" with the server.
Unfinished section
TODO: This section is not yet finished. It is missing descriptions for most routes, some error-code responses, etc.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#authorization","title":"Authorization","text":"
Bearer token, unless specified otherwise.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#errors","title":"Errors","text":"
The errors listed below are not exhaustive, and only include the most common errors associated with an endpoint. For rate limit errors, see the Rate Limits documentation.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#post-create-identity","title":"POST Create Identity","text":"
/.p2/core/v1/register
Creates an identity on a given server.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#request","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body","title":"Body","text":"
TODO: Re-evaluate if auth_payload is needed here.
Name Type Description
actor_name String The preferred name for this new identity.
auth_payload JSON-Object n. A. JSON
{\n \"actor_name\": \"alice\",\n \"auth_payload\": {\n \"password\": \"3c4589y70masfnmAML2\"\n }\n}\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#response","title":"Response","text":"201 Created409 Conflict Text Only
##### Body\n\n| Name | Type | Description |\n| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------- |\n| `fid` | String | The [Federation ID](../../Glossary.md#federation-id) of the new identity. |\n| `payload` :material-help:{title=\"This field is optional.\"} :material-code-braces:{title=\"The actual contents of this attribute are implementation-specific. polyproto-core does not provide any defaults for this field.\"} | JSON-Object | - |\n\n```json\n{\n \"fid\": \"xenia@example.com\",\n \"payload\": {\n \"some_account_information\": \"important information\",\n \"is_awesome\": true\n }\n}\n```\n
Text Only
Returned when the requested actor name is already taken within the namespace.\n\n##### Body\n\n```json\n{\n \"errcode\": 409,\n \"error\": \"P2CORE_FEDERATION_ID_TAKEN\"\n}\n```\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#post-identify","title":"POST Identify","text":"
/.p2/core/v1/session/identify
Identify on a foreign server and receive a session token.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#request_1","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_1","title":"Body","text":"Name Type Description
challenge String The completed challenge, consisting of a UTF-8 encoded signature value and the original challenge string. The signature value must have been created using the private identity key of that actor.
id_cert String, PEM, Base64 The client's ID-Cert, encoded in PEM & Base64.
auth_payload JSON-Object - JSON
{\n \"completed_challenge\": {\n \"challenge\": \"UH675678rbnFGNHJV2ijcnr3ghjHV74jah...\",\n \"signature\": \"Ac4hjv2ijcnr3ghjHV74jahUH675678rbnFGNHJV...\"\n },\n \"id_cert\": \"gA3hjv2ijcnr3ghjHV74jahUH675678rbnFGNHJV...\",\n \"auth_payload\": {\n \"my_custom_attribute\": \"my_custom_value\"\n }\n}\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#response_1","title":"Response","text":"201 Created"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_2","title":"Body","text":"Name Type Description
token String A session token, to be used for further identification/authentication
payload JSON-Object - JSON
{\n \"token\": \"G5a6hjv2ijcnr3ghjHV74jahUH675678rbnFGNHJV...\",\n \"payload\": {\n \"my_custom_response_attribute\": \"my_custom_response_value\"\n }\n}\n
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#put-revoke-session-authentication","title":"PUT Revoke session authentication","text":"
/.p2/core/v1/session/revoke
Revoke the current session's authentication by having the server invalidate the session token. Can also be seen as a \"logout\" operation.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#request_2","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_3","title":"Body","text":"
This request has no body.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#response_2","title":"Response","text":"204 No Content"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#body_4","title":"Body","text":"
This response has no body.
"},{"location":"APIs/auth/Routes%3A%20No%20registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/","title":"Authentication Routes: Registration needed","text":"
All API endpoints needed for implementing polyproto-auth. This Page only includes routes which a client can request from its home server. For routes which can also be accessed from a foreign server, or with no authentication at all, see the Client-Foreign Server API documentation
Unfinished section
TODO: This section is not yet finished. It is missing descriptions for most routes, some error-code responses, etc.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#authorization","title":"Authorization","text":"
Bearer token, unless specified otherwise.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#errors","title":"Errors","text":"
The errors listed below are not exhaustive, and only include the most common errors associated with an endpoint. For rate limit errors, see the Rate Limits documentation.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#post-authenticate-new-session","title":"POST Authenticate new Session","text":"
/.p2/core/v1/session/trust
Creates a new id_cert and a session token from a csr.
"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#request","title":"Request","text":""},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#body","title":"Body","text":"Name Type Description
actor_name String The actor name of the identity to authenticate as.
csr String, PEM A certificate signing request (CSR)
auth_payload JSON-Object n. A."},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#response","title":"Response","text":"201 Created"},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#body_1","title":"Body","text":"Name Type Description
id_cert String, PEM The ID-Cert for this unique Identity-Session combination
token String An authorization secret, called a \"token\", valid for this
id_cert."},{"location":"APIs/auth/Routes%3A%20Registration%20needed/#glossary","title":"Glossary","text":"
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Overviews/core/","title":"An Overview of polyproto","text":"
Work in Progress
This overview page is not yet finished. However, that what is there is already representative of what the polyproto protocol is about.
polyproto is a federated identity and message exchange protocol, which can be used for almost anything. If you'd like to build an application where federation, user control and data integrity are wanted, then polyproto is most likely for you. Read this overview to get to know the core concepts and technologies used in the protocol.
"},{"location":"Overviews/core/#identity","title":"Identity","text":"
Your identity is always represented by a Federation ID, FID for short. Conceptually, FIDs are nothing new, and they look like this:
xenia@some.example.com
Everything after the @ is your Home Servers' domain, and the part before the @ is your username. Together, this makes for an individual, yet globally unique identifier.
"},{"location":"Overviews/core/#certificates-and-keys","title":"Certificates and Keys","text":"
Identity Certificates - ID-Certs for short - represent your identity when logged in on different sessions. Each Identity Certificate contains the following information:
- Your federation ID, so that an account can be uniquely identified
- A session ID, which is unique for each session and does not change, even if the keys change
- An expiry date, after which the certificate becomes invalid
- A signature, generated by your home server, which acts as part of the proof that this certificate was actually issued by your home server
- Some information from your home server (Home server domain, certificate serial number)
- Information about the signature algorithm used
and, last but not least
- The public identity key of the client
For the sake of explanation, the most important parts here are the client public identity key, your federation ID, the home servers' domain and the home servers' signature for this certificate.
"},{"location":"Overviews/core/#message-signing","title":"Message signing","text":"
When you, for example, chat with someone on a different server, that other server is fully in control about what data it chooses to present to you. To make sure that this server is always telling you the truth, and not, for example, manufacturing chat messages or social media posts made by a person, messages are signed using a clients' public identity key.
flowchart LR\n hs[(Your Home Server)] --- you(You)\n you -- Send signed message --> fs[(A Foreign Server)]\n fs -- Forwards message --> other(Other User)\n fs --> verify{Verify message signature}\n other --> verify\n verify -- Get certificate from home server to verify --> hs
This is how it works:
- As touched on previously, every user client has an own identity key pair, comprised of a public and a private key. The public key is cryptographically linked to the private key, meaning that this public key can not belong to another private key. Signing data is done using the private key, which ONLY the client knows. Everyone can then use your public key to prove that this signature was generated by your client, and that the signature matches the data which was signed.
- Signatures are unique to a piece of data, meaning that two differing pieces of data signed by the same or different private keys will always1 produce different signatures. This is the case, even if the data only differs minutely (be it by a single space, or a single comma).
- Your home server attests to a clients' key pair, by creating a certificate for your public key, which it signs with its own secret, public/private key pair, and then sends to you. Your private key is never sent anywhere at all, and it does not need to be.
Now, your public identity key and your home servers' identity key are 'linked' to each other. This is represented in the ID-Cert you then receive from your home server.
- When communicating with another \"foreign\" server in polyproto, you first send that server your ID-Cert. The server can then prove the validity of your identity, simply by asking your home server for its public key and performing a quick signature verification.
- When sending data to the server, such as chat messages, your client computes the signature for that message using your private key, and attaches this signature to the message you send to other servers.
- Any user, at any point, can now take this signature, your identity certificate and your home servers' public key and cryptographically verify that it was, in fact, you who sent the message, and that the message was not tampered with in any way. To distribute the load of ID-Cert requests more evenly, it is always the duty of the server that the data exchange is happening on, to cache and hand out ID-Certs of users.
Info
If you are interested about the details, feel free to jump to section 7.1 in the protocol in the specification document, which covers this exact thing and more.
"},{"location":"Overviews/core/#trust","title":"Trust","text":"
Trusting the smallest possible amount of entities is great practice when it comes to security. polyproto makes sure that almost everyone you do trust is under constant scrutiny, and thus provides measures to verify a data authors' identity, and that the actual data has not been tampered with.
Aside from yourself, the entity with the most trust assigned to it is your home server. Creating your identity on a specific home server is a pledge from that server and its admins to you, where they promise not to create sessions on your behalf, or to otherwise perform actions which can be publicly identified to be carried out by you, without your explicit consent.
Should you ever change your mind about your home server's trustworthiness, you can always migrate to another server while keeping the ownership status of your data on all servers you have sent data to, even if your home server is offline indefinitely.
"},{"location":"Overviews/core/#multi-use","title":"Multi-use","text":"
polyprotos' API definitions and specification document intentionally leave space for implementation-specific data to be sent, where it makes sense. Nothing about the core protocol makes polyproto inherently unsuitable for any purpose.
"},{"location":"Overviews/core/#federation","title":"Federation","text":"
Federation in polyproto means using one identity or client to interact with multiple servers or even services at once. Implementing federation is straightforward, and entirely seamless to use for end users.
"},{"location":"Overviews/core/#technology","title":"Technology","text":"
Probably the most refreshing aspect about this new protocol is, that it is really boring. There is really nothing new about any given atomic aspect of polyproto. Polyproto uses well-known, tried and battle-tested technologies, such as asymmetric encryption, X.509-based public key infrastructure and -certificates, digital signatures, JSON over REST and other, already well established technologies such as WebSockets.
polyproto should be effortless - both for developers and for end users, who, ideally, should never have to notice any of the technical stuff going on in the background.
"},{"location":"Overviews/core/#conclusion","title":"Conclusion","text":"
This is just an outline about how polyproto works. The goal with this outline is to inform about the most relevant parts, while intentionally leaving out some details for the sake of clarity. If you have read and understood this overview, you should have no - or at least way less - trouble reading the full protocol specification, which covers a lot more details!
-
Signature/hash collisions, which although theoretically possible, are extraordinarily infrequent and thus, negligible in practical scenarios.\u00a0\u21a9
"},{"location":"Protocol%20Specifications/core/","title":"polyproto Specification","text":"
v0.1.0-alpha.1 - Treat this as an unfinished draft.
Semantic versioning v2.0.0 is used to version this specification. The version number specified here also applies to the API documentation.
- polyproto Specification
- 1. Terminology used in this document
- 2. Trust model
- 3. APIs and communication protocols
- 3.3 WebSockets
- 3.3.1 Events over REST
- 4. Federated identity
- 4.1 Authentication
- 4.1.1 Authenticating on a foreign server
- 4.1.2 Sensitive actions
- 4.2 Challenge strings
- 4.3 Protection against misuse by malicious home servers
- 5. Federation IDs (FIDs)
- 6. Cryptography and ID-Certs
- 6.1 Home server signed certificates for public client identity keys (ID-Cert)
- 6.1.1 Structure of an ID-Cert
- 6.1.1.1 Identity Descriptors (IDDs)
- 6.1.1.2 Extensions and constraints
- 6.1.1.3 Session IDs
- 6.1.2 Necessity of ID-Certs
- 6.1.3 Key rotation
- 6.1.4 Early revocation of ID-Certs
- 6.2 Actor identity keys and message signing
- 6.2.1 Message verification
- 6.2.2 Handling of external messages
- 6.3 Private key loss prevention and private key recovery
- 6.4 Caching of ID-Certs
- 6.4.1 Verifying that a newly retrieved ID-Cert is not out-of-date
- 6.5 Cryptographic recommendations
- 6.6 Best practices
- 6.6.1 Signing keys and ID-Certs
- 6.6.2 Home server operation and design
- 6.6.3 Private key loss prevention and private key recovery
- 7. Migrations
- 7.1 Identity migration
- 7.1.1 Redirects
- 7.2 Re-signing messages
- 7.2.1 Message batches
- 7.2.2 Server imposed limits
- 7.2.2.1 Body size
- 7.2.2.2 Interval between re-signing batches
- 7.3 Moving data
- 7.3.1 Content Addressing with relative roots
- 7.4 Challenges and trust
- 8. Protocol extensions (P2 extensions)
- 8.1 Extension design
- 8.2 Namespaces
- 8.3 Officially endorsed extensions
- 8.4 Versioning and yanking
- 8.4.1 Yanking
- 8.5 Dependencies
- 8.6 Routes
- 9. Services
- 9.1 Discoverability
- 9.1.1 Changing a primary service provider
The polyproto protocol is a home-server-based identity federation protocol specification intended for use in applications where actor identity is needed. polyproto focuses on federated identity, and does not specify any further application-specific features. It can be used standalone, as a method of authenticating across many applications and services, or as a base for federated protocol extensions and application implementations. The use of cryptography - namely digital signatures and X.509 certificates - make polyproto identities verifiable and portable. polyproto empowers actors, as the home server can be changed at any time, without losing data or connections to other actors.
This document is intended to be used as a starting point for developers wanting to develop software, which can operate with other polyproto implementations.
"},{"location":"Protocol%20Specifications/core/#1-terminology-used-in-this-document","title":"1. Terminology used in this document","text":"
The following terminology is used throughout this document:
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Protocol%20Specifications/core/#2-trust-model","title":"2. Trust model","text":"
polyproto operates under the following trust assumptions:
- Users entrust their home server and its admins with data security and discretion on actions appearing as actor-performed as, as with most home server based systems, it is possible for a home server to impersonate an actor in unencrypted communications.
- Impersonation can be detected by users, as home servers never have access to private keys of actors. To sign messages as an actor, a home server would have to use a different key pair.
- Users only trust information, which can be verified by cryptographic means. This includes verifying the identity of other actors and verifying the integrity of messages.
- In a federated context, users trust foreign servers with all unencrypted data they send to them.
- Foreign servers cannot impersonate users without immediate detection. Outsiders, meaning foreign servers and other actors, are unable to produce signatures that have a cryptographic connection to the actors' home server. This is assuming correct implementation of cryptographic standards, secure home server operation and non-compromised client devices, all of which are mostly out of the scope of this specification.
- Users rely on their home server for identity key certification, without the home server possessing the identity.
"},{"location":"Protocol%20Specifications/core/#3-apis-and-communication-protocols","title":"3. APIs and communication protocols","text":"
The polyproto specification defines a set of APIs. In addition to these REST APIs, polyproto employs WebSockets for real-time communication between clients and servers.
The APIs are divided into two categories:
- Routes: No registration needed: These routes are available to all clients, regardless of whether this server is the client's home server.
- Routes: Registration needed: These routes are only available to clients where the server is the client's home server.
All software aiming to federate with other polyproto implementations must implement the APIs defined in this specification. Implementations can choose to extend the APIs with additional routes, but must not remove or change the behavior of the routes defined in this specification.
"},{"location":"Protocol%20Specifications/core/#33-websockets","title":"3.3 WebSockets","text":"
WebSockets enable real-time communication between actor clients and servers.
WebSocket connections to polyproto servers consist of the following cycle:
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant g as Gateway\n\nc->>g: Establish connection\ng->>c: Recieve hello event\n\nloop TODO: interval\n c->>g: Send heartbeat event\n g->>c: Send heartbeat ACK Event\nend\n\nc->>g: Send identify payload\n\nalt Server accepts\n g->>c: Send ready event\nelse Server defined reason\n g->>c: Disconnect with specified reason\nend\n\n\nopt Resume connection#59;<br />otherwise, repeat from step 1\n c->>g: Open new connection\n c->>g: Send resume event\n g->>c: Send missed events\n g->>c: Send resumed event\nend\n
Fig. 1: Sequence diagram of a WebSocket connection to a polyproto server.
Info
To learn more about polyproto WebSockets and WebSocket Events, consult the WebSockets documentation.
"},{"location":"Protocol%20Specifications/core/#331-events-over-rest","title":"3.3.1 Events over REST","text":"
For some implementation contexts, a constant WebSocket connection might not be wanted. A client can instead opt to query an API endpoint to receive events, which would normally be sent through the WebSocket connection. Concrete polyproto-implementations and extensions can decide whether this alternative behavior is supported.
Example
An example of an implementation context where having a constant WebSocket might not be wanted would be Urban IoT devices, or devices with a limited or only periodically available internet connection.
Querying this endpoint yields a JSON-Array containing all events the session has missed since last querying the endpoint, or since last being connected to the WebSocket.
Depending on how many events the session has missed, the earliest events might be excluded from the response to limit the response bodies size. This behavior should be explicitly documented in implementations or extensions of polyproto.
Due to the intended use cases for retrieving events through REST rather than WebSockets, this endpoint is not a long-polling endpoint.
There are three intended, main modes for retrieving events in polyproto
- Keep a constant WebSocket connection whenever possible
- Keep a semi-constant WebSocket connection, perhaps connecting every x minutes for a set period of time
- Do not use WebSockets and only query the REST API
Polling a REST endpoint is inherently inefficient and therefore should only be done with a high interval, ranging from a few minutes to a few days. If a client requires information more often than that, then a WebSocket connection should be considered.
"},{"location":"Protocol%20Specifications/core/#4-federated-identity","title":"4. Federated identity","text":"
The federation of actor identities allows users to engage with foreign servers as if they were their home servers. For example, in polyproto-chat, an actor can send direct messages to users from a different server or join the Guilds of other servers.
Identity certificates defined in sections #6. Cryptography and ID-Certs and #6.1 Home server signed certificates for public client identity keys (ID-Cert) are employed to sign messages that the actor sends to other servers.
Using one identity for several polyproto implementations
An actor can choose to use the same identity for multiple polyproto implementations. Read section #9 for more information.
Info
You can read more about Identity Certificates in section #6.
"},{"location":"Protocol%20Specifications/core/#41-authentication","title":"4.1 Authentication","text":"
The core polyproto specification does not contain a strict definition of authentication procedures and endpoints. This allows for a wide range of authentication methods to be used. However, if implementations want to closely interoperate with each other, they should highly consider implementing the polyproto-auth standard for authenticating on home servers and foreign servers alike.
Warning
Close interoperation is only possible if all involved polyproto implementations have an overlapping set of supported authentication methods. Therefore, it is highly recommended to implement and use the polyproto-auth standard, unless your use case requires strictly requires a different authentication method. Of course, other authentication methods can be implemented in addition to polyproto-auth.
When successfully authenticated, a client receives a session token, which can then be used to access authenticated routes on the REST API and to establish a WebSocket connection. Each ID-Cert can only have one active session token at a time.
About session tokens
Session tokens are used to authenticate a user over a longer period of time, instead of, for example, requiring the user to solve a challenge string every time they want to access a protected route.
"},{"location":"Protocol%20Specifications/core/#411-authenticating-on-a-foreign-server","title":"4.1.1 Authenticating on a foreign server","text":"
Regardless of the authentication method used, the foreign server must verify the actor's identity before allowing them to perform any actions. This verification must be done by proving the cryptographic connection between an actors' home servers' public identity key and the actors' ID-Cert. Challenge strings, as described in Section 4.2 and in polyproto-auth are used for this purpose within this specification.
Servers must also check with the actors' home server to ensure that the ID-Cert has not been revoked. APIs for this purpose are defined in the API documentation.
"},{"location":"Protocol%20Specifications/core/#412-sensitive-actions","title":"4.1.2 Sensitive actions","text":"
Warning
Sensitive actions should require a second factor of authentication, apart from the actors' private key. This second factor can be anything from a password to TOTP or hardware keys, depending on the authentication method or standard used.
If this is not done, a malicious user who gained access to an actors' private key can lock that actor out of their account entirely, as the malicious user could revoke the actors' other ID-Certs, and thus prevent the actor from logging in again.
Sensitive actions include, but are not limited to:
- Generating a new ID-Cert
- Revoking an ID-Cert
- Changing the actors' federation ID
- Changing the actors' other factors of authentication
- Server administration actions
Clients should be prepared to gracefully handle the case where a sensitive action fails due to a lack of a second factor of authentication, and should prompt the user to provide the second factor of authentication.
"},{"location":"Protocol%20Specifications/core/#42-challenge-strings","title":"4.2 Challenge strings","text":"
Servers use challenge strings to verify an actor's private identity key possession, without revealing the private key itself. These strings, ranging from 32 to 256 UTF-8 characters, have a UNIX timestamp lifetime. If the current timestamp surpasses this lifetime, the challenge fails. The actor signs the string, sending the signature and their ID-Cert to the server, which then verifies the signature's authenticity.
All challenge strings and their responses created must be made public to ensure that a chain of trust can be maintained. A third party should be able to verify that the challenge string, which authorized a specific change in data, was signed by the correct private key. The API routes needed to verify challenges as an outsider are documented in the API documentation.
Tip
For public-facing polyproto implementations, it is recommended to use a challenge string length of at least 64 characters, including at least one character from each of the alphanumeric character classes ([a-zA-Z0-9]). Server implementations should ensure that challenge strings are unique per actor. If this is not the case, actors could potentially be the target of replay attacks.
Challenge strings can counteract replay attacks. Their uniqueness ensures that even identical requests have different signatures, preventing malicious servers from successfully replaying requests.
"},{"location":"Protocol%20Specifications/core/#43-protection-against-misuse-by-malicious-home-servers","title":"4.3 Protection against misuse by malicious home servers","text":"
To protect users from misuse by malicious home servers, a mechanism is needed to prevent home servers from generating federation tokens for users without their consent and knowledge.
Potential misuse scenario
A malicious home server can potentially request a federation token on behalf of one of its users, and use it to generate a session token on the actor's behalf. The malicious server can then impersonate the actor on another server, as well as read unencrypted data (such as messages, in the context of a chat application) sent on the other server.
Abstract
The above scenario is not unique to polyproto, and rather a problem other federated services/protocols, like ActivityPub, have as well. There is no real solution to this problem, but it can be mitigated a bit by making it more difficult for malicious home servers to do something like this without the actor noticing.
Polyproto servers need to inform users of new sessions. This visibility hampers malicious home servers, but does not solve the issue of them being able to create federation tokens for servers the actor does not connect to. This is because, naturally, users cannot receive notifications without a connection. Clients re-establishing server connections must be updated on any new sessions generated during their absence. The NEW_SESSION gateway event must be dispatched to all sessions, excluding the new session. The NEW_SESSION event's stored data can be accessed in the Gateway Events documentation.
Note
With proper safety precautions and strong encryption, it is extremely unlikely for a malicious server to be able to listen in on encrypted conversations, without all users in that conversation noticing. When implementing the polyproto-mls P2 extension, MLS's forward secrecy guarantees ensure that, in theory, a malicious session cannot decrypt any messages sent before its' join epoch. If secrecy or confidentiality are of concern, users should host their own home server and use end-to-end encryption, such as polyproto-mls.
"},{"location":"Protocol%20Specifications/core/#5-federation-ids-fids","title":"5. Federation IDs (FIDs)","text":"
Every client requires an associated actor identity. Actors are distinguished by a unique federation ID (FID). FIDs consist of their identifier, which is unique per instance, and the instance's root domain. This combination ensures global uniqueness.
FIDs used in public contexts are formatted as actor@optionalsubdomain.domain.tld, and are case-insensitive.
The following regular expression can be used to validate actor IDs: \\b([a-z0-9._%+-]+)@([a-z0-9-]+(\\.[a-z0-9-]+)*).
Info
The above regular expression is flavored for the Rust Programming Language, but can be easily adapted to other languages.
Note
Validating a federation ID with the above regex does not guarantee that the ID is valid. It only indicates that the federation ID is formatted correctly.
For all intents and purposes, a federation ID is a display of identity. However, verifying identity claims is crucial. See Section #6.1 and Section #6.2.2 for more information.
"},{"location":"Protocol%20Specifications/core/#6-cryptography-and-id-certs","title":"6. Cryptography and ID-Certs","text":""},{"location":"Protocol%20Specifications/core/#61-home-server-signed-certificates-for-public-client-identity-keys-id-cert","title":"6.1 Home server signed certificates for public client identity keys (ID-Cert)","text":"
The ID-Cert, a X.509 certificate, validates a public actor identity key. It is an actor-generated CSR (Certificate Signing Request), signed by a home server, encompassing actor identity information and the client's public identity key. Clients can get an ID-Cert in return for a valid and well-formed CSR. Generating a new ID-Cert is considered a sensitive action and therefore should require a second factor of authentication.
A CSR in the context of polyproto will be referred to as an ID-CSR. ID-CSRs are DER- or PEM-encoded PKCS #10 CSRs, with a few additional requirements.
All ID-Certs are valid X.509 v3 certificates. However, not all X.509 v3 certificates are valid ID-Certs.
ID-Certs form the basis of message signing and verification in polyproto. They are used to verify the identity of a client, and to verify the integrity of messages sent by a client.
An ID-CSR includes the following information, according to the X.509 standard:
- The public identity key of the client.
- An identity descriptor (IDD), describing the actor the certificate is issued to. The IDD must be formatted according to Section 6.1.1.1.
- The signature algorithm used to sign the certificate.
- The signature of the certificate, generated by using the entities' private identity key.
When signing an ID-CSR, the home server must verify the correctness of all claims presented in the CSR.
Important
All entities receiving an ID-Cert MUST inspect the certificate for correctness and validity. This includes checking whether the signature matches the certificates' contents and checking the certificate's validity period.
Actors must use a separate ID-Cert for each client or session they use. Separating ID-Certs limits the potential damage a compromised ID-Cert can cause.
For two implementations of polyproto to be interoperable, they must support an overlapping set of digital signature algorithms. See Section 6.5 for more information on cryptographic recommendations.
"},{"location":"Protocol%20Specifications/core/#611-structure-of-an-id-cert","title":"6.1.1 Structure of an ID-Cert","text":"
The ID-Cert is a valid X.509 certificate, and as such, it has a specific structure. The structure of an X.509 certificate is defined in RFC5280. ID-Certs encompass a subset of the structure of an X.509 certificate.
ID-Certs have the following structure:
Field Description Special requirements, if any X.509 equivalent Correctly formatted Name attribute, according to #6.1.1.1 Identity descriptor Issuer Name A unique identifier for the certificate, used by the CA to identify this certificate. Must be unique across all certificates issued by a home server. Serial Number The algorithm used to sign the certificate. Certificate Signature Algorithm & Signature Algorithm ID The signature of the certificate, generated by using the home servers' private identity key. Certificate Signature The expiry date of the certificate. Time must not be after expiry date of the home server's root certificate Validity period: Not After Certificate validity period starting date Time must not be before the home server's root certificate was generated Validity period: Not Before X.509 Version Number (v3) polyproto only uses Version 3 X.509 certificates. Version Number The public identity key of the client. Subject Public Key Info: Subject Public Key The public key algorithm used to generate the client's public identity key. Subject Public Key Info: Public Key Algorithm The session ID of the client. No two valid certificates for one session ID can exist. Session IDs have to be unique per user. Subject Unique Identifier Extensions Extensions and Constraints Extensions"},{"location":"Protocol%20Specifications/core/#6111-identity-descriptors-idds","title":"6.1.1.1 Identity Descriptors (IDDs)","text":"
polyproto Identity Descriptors are a subset of the X.509 certificate's distinguished name. Distinguished Names (DNs), according to the LDAP Data Interchange Format (LDIF). The DN is a sequence of relative distinguished names (RDNs).
The identity descriptor must be unique for each certificate issued by a home server. The DN of an ID-Cert must meet all of the following requirements:
- Identity descriptor (IDD) must have \"common name\" attribute. If the ID-Cert is for an actor, the common name must be the local name of the actor. In the case of an actor with an FID of
xenia@example.com, the \"common name\" would be xenia. If the ID-Cert is a self-signed home server certificate, the \"common name\" attribute must not be present. - Must have at least one domain component, specifying the home servers' FQDN (fully qualified domain name).
- If the ID-Cert or ID-CSR is for an actor, the IDD must include the
UID (OID 0.9.2342.19200300.100.1.1) and uniqueIdentifier (OID 0.9.2342.19200300.100.1.44) fields. UID is the federation ID of the actor, e.g. actor@fqdn-of-home server.example.com.uniqueIdentifier is a Session ID.
- Can have other attributes, if the additional attributes do not conflict with the above requirements. Additional attributes might be ignored by other home servers and other clients, unless specified otherwise in a polyproto extension. Additional attributes, which are not part of a polyproto extension must be non-critical X.509 extensions.
If the home server does not have a subdomain or top level domain, the dc fields for these components should be omitted.
"},{"location":"Protocol%20Specifications/core/#6112-extensions-and-constraints","title":"6.1.1.2 Extensions and constraints","text":"
The following constraints must be met by ID-Certs:
- If the ID-Cert is a root certificate
- It must have the
CA flag set to true. The path length constraint must be present and set to 0. - It must have the
keyCertSign key usage flag set to true.
- If the ID-Cert is an actor certificate
- It must have the
CA flag set to false or omitted. - It must have the
keyCertSign key usage flag set to false or omitted. - It must have the
digitalSignature key usage flag OR contentCommitment flags set to true.
Key Usage Flags and Basic Constraints are critical extensions. Therefore, if any of these X.509 extensions are present, they must be marked as \"critical\". ID-Certs not adhering to this standard must be treated as malformed.
"},{"location":"Protocol%20Specifications/core/#6113-session-ids","title":"6.1.1.3 Session IDs","text":"
The session ID is an ASN.1 Ia5String chosen by the actor requesting the ID-Cert. It is used to uniquely identify a session. The session ID must be unique for each certificate issued to that actor. A session ID can be re-used if the session belonging to that session ID has become invalid. Session ID re-use in this case also applies, when a different ID-Cert wants to use the same session ID, provided that the session ID is not currently in use. If the session ID is currently in use, the actor requesting the ID-Cert must select a different session ID, as session IDs must not be overridden silently.
Session IDs are 1 - 32 characters long and. They can contain any character permitted by the ASN.1 IA5String type.
Session IDs can be used to identify a session across devices, or to detect if a new, perhaps malicious session has been created.
"},{"location":"Protocol%20Specifications/core/#612-necessity-of-id-certs","title":"6.1.2 Necessity of ID-Certs","text":"
The addition of a certificate is necessary to prevent a malicious foreign server from abusing public identity key caching to impersonate an actor. Consider the following example, which employs foreign server public identity key caching, but no home server issued identity key certificates:
Potential misuse scenario
A malicious foreign server B can fake a message from Alice (Home server: Server A) to Bob (Home Server: Server B), by generating a new identity key pair and using it to sign the malicious message. The foreign server then sends that message to Bob, who will then request Alice's public identity key from Server B, who will then send Bob the malicious public identity key. Bob will succeed in verifying the signature of the message, and not notice that the message has been crafted by a malicious server.
The above scenario is not possible with home server issued identity key certificates, as the malicious server cannot generate an identity key pair for Alice, which is signed by Server A.
"},{"location":"Protocol%20Specifications/core/#613-key-rotation","title":"6.1.3 Key rotation","text":"
A session can choose to rotate their ID-Cert at any time. This is done by generating a new identity key pair, using the new private key to generate a new CSR, and sending the new Certificate Signing Request to the home server, along with at least one new KeyPackage and a corresponding 'last resort' KeyPackage, if encryption is offered. The home server will then generate the new ID-Cert, given that the CSR is valid and that the server accepts the creation of new ID-Certs at this time.
Rotating keys is done by using an API route, which requires authorization.
Note
Sessions can request a new ID-Cert for any session of the same actor. Most other, currently existing services also allow for this, as it is a common use case for user to want to, perhaps, log out of devices they no longer use. Depending on your use case, this might be a security concern. Whether and how this risk is mitigated is up to concrete implementations.
Home servers must keep track of the ID-Certs of all users (and their clients) registered on them, and must offer a clients' ID-Cert for a given timestamp on request. This is to ensure messages sent by users, even ones sent a long time ago, can be verified by other servers and their users. This is because the public key of an actor likely changes over time and users must sign all messages they send to servers. Likewise, a client should also keep all of its own ID-Certs stored perpetually, to potentially verify its identity in case of a migration.
Users must hold on to all of their past key pairs, as they might need them to migrate their account in the future. How this is done is specified in section 6.3: Private key loss prevention and private key recovery.
The lifetime of an actor ID-Cert should be limited to a maximum of 60 days. This is to ensure that even in a worst-case scenario, a compromised ID-Cert can only be used for a limited amount of time. The renewal of an ID-Cert is considered a sensitive action and should require a second factor of authentication. A client that has this second factor of authentication stored should renew the ID-Cert of the authenticated actor without further interaction.
Server ID-Certs should be rotated way less often (every 1-3 years). Only rotate a server ID-Cert if it is suspected to be compromised, is lost, or has expired.
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant s as Server\n\nc->>c: Create CSR for own identity key\nc->>s: Request key rotation/CSR signing, CSR attached\ns->>s: Verify validity of claims presented in the CSR\nalt verify success\n s->>s: Create ID-Cert for Client\n s->>c: Respond with ID-Cert\nend
Fig. 2: Sequence diagram depicting the process of a client that uses a CSR to request a new ID-Cert from their home server.
A server identity key's lifetime might come to an early or unexpected end, perhaps due to some sort of leak of the corresponding private key. When this happens, the server should generate a new identity key pair and broadcast the SERVER_KEY_CHANGE gateway event to all clients. Clients must request new ID-Certs through a CSR. Should a client be offline at the time of the key change, it must be informed of the change upon reconnection.
"},{"location":"Protocol%20Specifications/core/#614-early-revocation-of-id-certs","title":"6.1.4 Early revocation of ID-Certs","text":"
A note about CRLs
It is common for systems relying on X.509 certificates for user authentication to use Certificate Revocation Lists (CRLs) to keep track of which certificates are no longer valid. This is done to prevent a user from using a certificate that has been revoked.
CRLs are difficult to implement well, often requiring many resources to keep up to date, and are also not always reliable. OCSP (Online Certificate Status Protocol) is a more modern, reliable and easier to implement alternative. Still, it potentially requires many resources to keep up with demand, while introducing potential privacy concerns.
polyproto inherently mitigates some of the possible misuse of a revoked certificate, as the validity of a certificate is usually checked by many parties. Especially, if the revocation process is initiated by the actor themselves, the actor already lets all servers they are connected to know that the certificate in question is no longer valid.
polyproto does not require the use of CRLs or OCSP.
An ID-Cert can be revoked by the home server or the actor at any time. This can be done for various reasons, such as a suspected leak of the private identity key.
When an ID-Cert is revoked, the server must revoke the session associated with the revoked ID-Cert. Revoking an ID-Cert is considered a sensitive action and therefore should require a second factor of authentication.
Info
The above paragraph is true for both foreign and home servers. The API routes associated with revoking an ID-Cert are the same regardless of the server type.
Revocation detection
For information on how revocation detection is supposed to be handled, concern the excerpt \"Caching ID-Certs and cache TTLs\" TODO fix link
TODO: Write about identifier changing and how to handle that across servers TODO: Perhaps recommend never using more than a specified number of certificates at once to make re-signing easier
"},{"location":"Protocol%20Specifications/core/#62-actor-identity-keys-and-message-signing","title":"6.2 Actor identity keys and message signing","text":"
As briefly mentioned section #4, users must hold on to an identity key pair at all times. This key pair is used to represent an actor's identity and to verify message integrity, by having an actor sign all messages they send with their private identity key. The key pair is generated by the actor. An actor-generated identity key certificate signing request (CSR) is sent to the actor's home server when first connecting to the server with a new session, or when rotating keys. The key is stored in the client's local storage. Upon receiving a new identity key CSR, a home server will sign this CSR and send the resulting ID-Cert to the client. This certificate is proof that the home server attests to the clients key. Read section 6.1 for more information about the certificate.
The private key from the key pair that the server has generated an ID-Cert for will be used to create digital signatures for the contents of all messages sent by this session. This digital signature must be attached to the message itself, so that other actors can verify the integrity of the message contents.
Info
polyproto does not define what messages themselves look like, apart from this hard requirement. The format of a message is up to polyproto extensions and implementations to define.
"},{"location":"Protocol%20Specifications/core/#621-message-verification","title":"6.2.1 Message verification","text":"
To ensure message integrity through signing, clients and servers must verify message signatures. This involves cross-checking the message signature against the sender's ID-Cert and the senders' home server's ID-Cert, while also confirming the validity of the ID-Cert attached to the message and ensuring its public key matches the sender's.
Info
Signature verification must always be \"strict\", meaning that signature schemes producing malleable signatures and weak public keys must be rejected.
Example
Say we have two actors. Alice, who is registered on Server A, and Bob, who is registered on Server B. Alice and Bob are having a conversation on Server B. Given a signed message from Alice, such as Bob would receive from Server B, the process of verifying the signature would look like this:
sequenceDiagram\nautonumber\n\nactor b as Bob\nparticipant sb as Server B\nparticipant sa as Server A\n\nsb->>b: Alice's signed message\nopt Server A's ID-Cert is not cached on Bob's client\n b->>sa: Request Server A ID-Cert\n sa->>b: Server A ID-Cert\nend\nopt Alice's ID-Cert is not cached on Bob's client\n b->>sb: Request Alice's ID-Cert\n opt Alice's ID-Cert is not cached on Server B\n sb->>sa: Request Alice's ID-Cert\n sa->>sb: Alice's ID-Cert\n end\n sb->>b: Alice's ID-Cert\nend\nb->>b: Verify signature of Alice's message (Fig. 4)
Fig. 3: Sequence diagram of a successful message signature verification.
Abstract
You should read about the details of ID-Cert lookup load distribution via caching and why Bob should first try to request Alice's certificate from Server B instead of Alice's home server (Server A) in the corresponding section of this protocol specification. Understanding both sections is crucial for building secure, scalable and compliant implementations of polyproto.
TODO: IDEA: To keep other servers from not re-requesting the idcert after the ttls has passed, the idcert should have some sort of timestamp that is signed by the original server, so that clients can verify that a server has the most up-to-date idcert cached for a user -flori
Info
A failed signature verification does not always mean that the message is invalid. It may be that the actor's identity key has changed, and that Server B has not yet received the new public identity key for some reason. However, if the signature cannot be verified at a certain time, this information must be communicated to the actor performing the verification.
"},{"location":"Protocol%20Specifications/core/#622-handling-of-external-messages","title":"6.2.2 Handling of external messages","text":"
In the context of federation with other federation protocols, such as ActivityPub, it is possible for actors to receive messages, which do not have a signature attached to them. If a P2 extension explicitly allows for this, it is possible for a polyproto server to forward such messages to clients. If a P2 extension does not explicitly allow for this, both servers and clients must reject such messages. Clients receiving unexpected external messages should inform the actor about the fact that a server has tried to send them an invalid, possibly malicious message.
Before a polyproto server forwards such a message to clients, it must add an \"external\" property to the message object. If possible in the data format used, this property should be set to a boolean value of true. If the data format does not support boolean values, the property should be set to a string value of true in all lowercase characters. This property must be passed along to the client or clients receiving the message.
If the actor receiving this external message is human, the client must inform the actor that the message is external, and that the message has not been signed by the sender. External messages should be distinguishable from signed messages at first glance.
"},{"location":"Protocol%20Specifications/core/#63-private-key-loss-prevention-and-private-key-recovery","title":"6.3 Private key loss prevention and private key recovery","text":"
As described in previous sections, actors must hold on to their past identity key pairs, should they want or need to migrate their account.
Home servers must offer a way for actors to upload and recover their private identity keys while not having access to the private keys themselves. Private identity keys must be encrypted with strong passphrases and encryption schemes such as AES, before being uploaded to the server. Authenticated actors can download their encrypted private identity keys from the server at any time. All encryption and decryption operations must be done client-side.
If any uncertainty about the availability of the home server exists, clients should regularly download their encrypted private identity keys from the server and store them in a secure location. Ideally, each client should immediately download their encrypted private identity keys from the server after connecting. Clients should never store key backups in an unencrypted manner.
Whether an actor uploads their encrypted private identity keys to the server is their own choice. It is also recommended backing up the encrypted private identity keys in some other secure location.
The APIs for managing encrypted private identity keys are documented in the API documentation.
- Upload encrypted private key material
- Get encrypted private key material
- Delete encrypted private key material
- Get encrypted private key material upload size limit
"},{"location":"Protocol%20Specifications/core/#64-caching-of-id-certs","title":"6.4 Caching of ID-Certs","text":"
The caching of ID-Certs is an important mechanism in polyproto to aid in fairly distributing the load generated by ID-Cert lookups to the servers generating the traffic, not to the server the ID-Cert is actually from. This practice should help make the operation of low-resource home servers, used exclusively for hosting identities, more viable.
This section of the protocol definition defines required behaviors related to the correct caching of ID-Certs for both home servers and clients.
To make this section more understandable, we will bring back the example from section 6.2.1:
Revisiting the example scenario from section 6.2.1
Example
Say we have two actors. Alice, who is registered on Server A, and Bob, who is registered on Server B. Alice and Bob are having a conversation on Server B. Given a signed message from Alice, such as Bob would receive from Server B, the process of verifying the signature would look like this:
sequenceDiagram\nautonumber\n\nactor b as Bob\nparticipant sb as Server B\nparticipant sa as Server A\n\nsb->>b: Alice's signed message\nopt Server A's ID-Cert is not cached on Bob's client\n b->>sa: Request Server A ID-Cert\n sa->>b: Server A ID-Cert\nend\nopt Alice's ID-Cert is not cached on Bob's client\n b->>sb: Request Alice's ID-Cert\n opt Alice's ID-Cert is not cached on Server B\n sb->>sa: Request Alice's ID-Cert\n sa->>sb: Alice's ID-Cert\n end\n sb->>b: Alice's ID-Cert\nend\nb->>b: Verify signature of Alice's message (Fig. 4)
Fig. 3: Sequence diagram of a successful message signature verification.
In the case where alice@server-a.example.com and bob@server-b.example.com are having a conversation where the communications server is any server other than server-a.example.com, Bob should request Alice's ID-Cert from that server first, instead of from server-a.example.com.
Further notes on why we consider this cached distribution process a good idea
Bob's client could request Alice's public identity key from Server A, instead of Server B. However, this is discouraged, as it
- Generates unnecessary load on Server A; Doing it this way distributes the load of public identity key requests more fairly, as the server that the message was sent on is the one that has to process the bulk of public identity certificate requests.
- Would expose unnecessary metadata to Server A; Server A does not need to know who exactly Alice is talking to, and when. Only Server B, Alice and Bob need to know this information. Always requesting the public identity key from Server A might expose this information to Server A.
Clients should only use Server A as a fallback for public identity key verification, if Server B does not respond to the request for Alice's public identity key, or if the verification fails with the public identity key from Server B. Security considerations listed in this section of the protocol definition ensure that this cached distribution process is safe and trustworthy
Both Bob's client and Server B should now cache Server A's and Alice's ID-Certs, to avoid having to request them again.
The TTL (time to live) of these cached items should be relatively short. Recommended values are between one (1) and twelve (12) hours. Cached ID-Certs must be evicted from the cache, after the TTL has expired. Expired cached ID-Certs must not be used for signature verification of new messages, even if the client cannot renew its cache. All of this applies to both servers and clients. The TTL for a certificates' cache duration is dictated by the home server, which that certificate has been issued by. You can read more on that in subsection 1 of this section.
Why not select longer lived TTLs for cached ID-Certs?
Suppose that an actors' private identity key is compromised. The actor notices this, and revokes their ID-Cert. If the TTL of cached ID-Certs is too long, the compromised ID-Cert might still be used for signature verification for a long amount of time, even after the ID-Cert has been revoked. This is a problem in the following hypothetical scenario with malicious actor \"Eve\" and victim \"Alice\":
Downside of using higher values for a TTL
- One of Alice's private identity keys is compromised.
- Malicious actor Eve logs onto Server X, which Alice has never connected to before, using Alice's ID-Cert of which the corresponding private identity key has been compromised.
- In the meantime, Alice notices the breach, requesting the revocation of her ID-Cert on all servers she is connected to.
- Server X does not get this revocation message, as Alice does not know about her connection to Server X, where Eve is impersonating Alice.
- Eve can now impersonate Alice on Server X for as long as the TTL of the cached ID-Cert on Server X has not expired. With a high value, this could be a long time.
If the verification fails, Bob's client should try to re-request the key from Server B first. Should the verification fail again, Bob's client can try to request Alice's public identity key and ID-Cert from Server A (Alice's home server). The signature verification process should then be re-tried. Should the verification still not succeed, the message should be treated with extreme caution.
sequenceDiagram\nautonumber\n\nactor b as Bob\nparticipant sb as Server B\nparticipant sa as Server A\n\nb->>b: Verify signature of Alice's message, attempt 1\nalt Verification fails\n b->>sb: Request Alice's ID-Cert\n sb->>b: Alice's ID-Cert\n b->>b: Verify signature of Alice's message, attempt 2\n opt Verification fails again\n b->>sa: Request Alice's ID-Cert\n sa->>b: Alice's ID-Cert\n b->>b: Verify signature of Alice's message, final attempt\n opt Verification is still unsuccessful\n b-->b: Treat Alice's message with extreme caution.\n end\n end\nelse Verification succeeds\n b-->b: Treat Alice's message as verified.\nend
Fig. 4: Sequence diagram showing how message verification should be handled if the first attempt to verify the signature fails.
After evicting a cached ID-Cert:
- A client should request an up-to-date ID-Cert of the target actor from the server where the actor was last seen by the client.
- A server should request an up-to-date ID-Cert from the target actors' home server.
Info
It is not of vital importance that a client requests an ID-Cert of an actor whose ID-Cert has just been evicted from the cache from the server, where the actor was last seen by the client precisely. This means, that a client application doesn't necessarily need to update an internal state of where that actor has last been seen every single time that actor sends a message somewhere. This internal state update could instead happen every 5, 30, or even 60 seconds. What is important, however, is that this state update does eventually happen within a reasonable amount of time, to help achieve the goal of dynamic server load distribution.
"},{"location":"Protocol%20Specifications/core/#641-verifying-that-a-newly-retrieved-id-cert-is-not-out-of-date","title":"6.4.1 Verifying that a newly retrieved ID-Cert is not out-of-date","text":"
While the goal of achieving dynamic server load distribution to increase the viability of small, low-resource home servers is a noble one, this goal must not undermine P2s trust model, which other aspects of the protocol work very hard to uphold. Retrieving ID-Certs from a middleman introduces a new attack surface which must be mitigated. Consider the following example:
Example attack abusing blind middleman trust
- One of Alice's private identity keys is compromised.
- Malicious actor Eve logs onto a malicious Server X which is controlled by Eve, impersonating Alice by using Alice's ID-Cert of which the corresponding private identity key has been compromised.
- In the meantime, Alice notices the breach, requesting the revocation of her ID-Cert on all servers she is connected to.
- Server X does not care for this revocation message, as it is malicious (attacker controlled)
- Eventually, the TTL for this compromised certificate expires. Users on Server X contact the server for the latest certificate of Alice.
- Server X responds with the compromised ID-Cert, claiming that this is the most up-to-date ID-Cert, even though it has been revoked.
- Through all users trusting Server X blindly, Eve and Server X can impersonate Alice for as long as Alice's compromised ID-Cert would have been valid for (valid-not-after attribute in X.509 certificates). Until then, users do not notice that this certificate has been revoked and should no longer be valid.
This kind of attack has been considered and mitigated in polyproto. bla
"},{"location":"Protocol%20Specifications/core/#65-cryptographic-recommendations","title":"6.5 Cryptographic recommendations","text":"
For two implementations of polyproto to be interoperable, they must support an overlapping set of digital signature algorithms.
If technically practical, all implementations of polyproto must support use of the Ed25519 digital signature algorithm for signing messages and generating ID-Certs. The use of the RSA algorithm for digital signatures is heavily discouraged.
"},{"location":"Protocol%20Specifications/core/#66-best-practices","title":"6.6 Best practices","text":"
The following subsections are dedicated to documenting best practices to consider when implementing polyproto.
"},{"location":"Protocol%20Specifications/core/#661-signing-keys-and-id-certs","title":"6.6.1 Signing keys and ID-Certs","text":"
- When a server is asked to generate a new ID-Cert for an actor, it must make sure that the CSR is valid and, if set, has an expiry date less than or equal to the expiry date of the server's own ID-Cert.
- Due to the fact that a
SERVER_KEY_CHANGE gateway event is bound to generate a large amount of traffic, servers should only manually generate a new identity key pair when absolutely necessary and instead select a fitting expiry date interval for their ID-Certs. It might also be a good idea to stagger the sending of SERVER_KEY_CHANGE gateway events, to prevent a server from initiating a DDoS attack on itself. - When a client or server receives the information that an actor's client identity key has been changed, the client/server in question should update their cached ID-Cert for the actor in question, taking into account the session ID of the new identity key pair.
"},{"location":"Protocol%20Specifications/core/#662-home-server-operation-and-design","title":"6.6.2 Home server operation and design","text":"
- Use a caching layer for your home server to handle the potentially large amount of requests for ID-Certs without putting unnecessary strain on the database.
"},{"location":"Protocol%20Specifications/core/#663-private-key-loss-prevention-and-private-key-recovery","title":"6.6.3 Private key loss prevention and private key recovery","text":"
- It is a good idea for home servers to limit the upload size and available upload slots for encrypted private identity keys.
"},{"location":"Protocol%20Specifications/core/#7-migrations","title":"7. Migrations","text":"
polyproto empowers the end-user by defining straightforward mechanisms to change their home server while preserving their identity, moving messages to another server, or both.
Identity migration allows actors to transparently re-assign ownership of their identity and messages to a new identity. This allows actors to switch home servers while not losing ownership of messages sent by them.
Message migration allows actors to move messages from one service-provider to another in a tamper-resistant way. This makes it possible for actors to switch service providers, taking some or all of their messages with them. Which messages can be moved is up to P2 extensions to define, as it might not always be possible to move all messages. Some messages might be tied to a specific context, which is unavailable on the new server.
Example: Information tied to a specific context
In a chat application, there might exist a group chat with a lot of people in it. Moving your messages from this group chat to another server might be impossible, depending on the architecture of the chat application. Typically, the messages in a group chat are stored on the server hosting the group. Moving the messages of one individual from one server to another is not possible in these cases.
Example: Information not necessarily tied to a specific context
Continuing the chat application example, it might very well be possible to move messages written in a private chat between two actors from one server to another. An examplary architecture where this is possible, is where all private messages are stored on the server of the actor who sent the message. Here, an actor can move their messages to another server without any issues.
Migrating an actor always involves reassigning the ownership of all actor-associated data in the distributed network to the new actor. Should the old actor want to additionally move all data from the old home server to another home server, more steps are needed. Account migration is not considered a sensitive action.
This chapter defines behaviors and security mechanisms associated with migrating an actor identity or messages.
"},{"location":"Protocol%20Specifications/core/#71-identity-migration","title":"7.1 Identity migration","text":"
Transferring message ownership from an old to a new account, known as identity migration, necessitates coordination between the two involved accounts.
Identity migration is a process which can be broken down into the following steps:
- Setting up a redirect
- Re-signing data
It is not required that the new account is located on another home server as the old account. Re-signing data and setting up a redirect are both not mandatory steps. It is up to actors to decide to which extent they wish to perform the migration.
"},{"location":"Protocol%20Specifications/core/#711-redirects","title":"7.1.1 Redirects","text":"
Setting up a redirect is an optional step in the identity migration process, helping make the transition from the old account to the new account smoother.
A redirect has to be confirmed by both the redirection source and the redirection target. The redirect is only valid for one specific redirection target. Redirection targets must be valid actors and their home servers must be reachable when the redirect is being set up.
Info
\"Optional\" does not mean that home servers can choose to not implement this feature. Instead, it means that actors can choose to not use this feature.
sequenceDiagram\nautonumber\n\nactor aa as Alice Old (Redirection source)\nparticipant sa as \"Alice Old\" Home Server\nactor ab as Alice New (Redirection target)\n\nNote over aa, ab: These steps may be done in any order<br/>and are not necessarily sequential\npar Verifying redirect intent by passing key trial\n aa->>sa: Request redirect to Alice New\n sa-)sa: Confirm \"Alice New\"<br/>is valid actor by resolving FID \n sa->>aa: List of keys to<br/>verify + challenge string\n aa->>sa: Completed challenge<br/>for each key on the list\n sa->>sa: Set redirect status to<br/>\"confirmed by redirection source\"\nand Notifying the redirection source's home server of the redirection target\n ab->>sa: Request redirect from Alice Old\n sa->>sa: Verify authenticity of Alice New's identity by verifying ID-Cert\n note over sa: Alice New's ID-Cert is determined to be valid\n sa->>ab: Challenge string (See section 4.1.1:<br/>Authenticating on a foreign server)\n ab->>sa: Completed challenge\n sa->>sa: Set redirect status to<br/>\"confirmed by redirection target\"\nend\nsa->>sa: Check, if both redirection source and target have confirmed the redirect\nalt If both redirection source and target have confirmed the redirect\n sa->>sa: Use HTTP 307 to redirect all requests for<br/>Alice Old to Alice New\nelse\n Note over sa: Do nothing\nend
Fig. 5: Sequence diagram depicting the setting up of a redirect.
Until a redirection source actor deletes their account, the home server of that actor should respond with 307 Temporary Redirect to requests for information about the redirection source. After the redirection source deletes their account, Server A can select to either respond with 308 Permanent Redirect, or to remove the redirect entirely.
"},{"location":"Protocol%20Specifications/core/#72-re-signing-messages","title":"7.2 Re-signing messages","text":"
Re-signing messages is the process of transparently changing the signature of messages while leaving the content of the messages unchanged. \"Transparently\" refers to the fact that an outsider can verify the following facts:
- Both involved actors have agreed to the re-signing of the messages
- The \"old\" actor has proven ownership of the signature keys used to produce the \"old\" signatures of the messages
- The message content has not changed during the re-signing process
The intended use cases for re-signing messages are:
- Changing ownership of messages from one actor to another. This enables seamless transitions between accounts, while preserving the integrity of the messages.
- Reducing the amount of keys that need to be remembered by an actor, done if the actor deems it to be convenient.
- \"Rotate keys of past messages\" - This is useful when an actor's private identity key has been compromised, and the actor wants to ensure that all messages sent by them are still owned by them and not at risk of being tampered with.
Actors must not be able to re-sign messages, to which they cannot prove signature-key ownership of.
Additionally, servers must verify the following things about re-signed messages:
- The new signature matches the messages' contents, and is valid
- The ID-Cert corresponding to the new signature is a valid ID-Cert, issued by the correct home server
- The ID-Cert corresponding to the new signature has a public key that was specified in the
allowedResigningKeys property sent to the server when message re-signing was requested. - The contents of the message have not been changed during the re-signing process
- The
expires UNIX timestamp, specified when the server replied to the re-signing request, has not been reached or passed when the re-signed message was received by the server.
Below is a sequence diagram depicting a typical re-signing process, which transfers ownership of messages from Alice A to Alice B.
sequenceDiagram\nautonumber\n\nactor aa as Alice A\nactor ab as Alice B\nparticipant sc as Server \"C\" with stored<br/>messages from Alice A\n\naa->>sc: Request allow message re-signing for Alice B + list of \"allowed\" pubkeys\nsc->>aa: List of keys to verify + challenge string (Key trial)\naa->>sc: Completed challenge for each key on the list\nsc->>sc: Verify challenge, unlock re-signing for Alice B (only \"allowed\" pubkeys)\nsc->>aa: Re-signing of messages for Alice B allowed\nloop Do, while there are messages left to be re-signed\n ab->>sc: Request message re-signing<br/>for Alice A's messages\n sc->>ab: Batch of old messages,<br/>including the signatures + actor certificates\n Note over ab: The client should fetch missing information<br/>such as missing ID-Certs or server public keys<br/>needed to validate the messages from the<br/>corresponding servers, if applicable\n ab->>ab: Verify that Server C has not<br/>tampered with messages by<br/>checking old signatures with own keys\n ab->>ab: Re-sign messages with own keys\n ab->>sc: Send new messages\n sc->>sc: Verify that only FID and signature related fields have changed\n sc->>sc: Verify that key used to produce signature is on \"allowed\" list\n sc->>ab: Acknowledge successful re-signing of batch\n opt\n ab--)ab: Pause for arbitrary amount of time\n end\nend
To allow for a singular set of behaviors, which fit the three intended use cases mentioned prior, not all messages stored by the server of an actor need to be re-signed. Besides querying for all non-re-signed messages, actors can also query or all non-resigned messages, whose signatures correspond to a specific ID-Cert or set of ID-Certs. The API routes for re-signing messages are documented in the API documentation.
"},{"location":"Protocol%20Specifications/core/#721-message-batches","title":"7.2.1 Message batches","text":"
Messages, which have not yet been re-signed are being delivered to an actor in batches. A batch is a JSON object, representing messages sent, using the same ID-Cert. An exemplary array of message batches, as returned by the server, might look as follows:
JSON
[\n {\n id_cert: \"QLASDiohs79034sjkldfny8eppqxncp7n4g9vozeyuiwofxb...\",\n messages: [\n {\n signature: \"ASDiohs79034sjkldfny8eppqxncp7n4g9vozeyuiwofxb...\",\n content: {\n message: \"Hello!\"\n }\n },\n {\n signature: \"ASDiohs7902347sjkldfny8eafhjhjafdlk4g121ghjkz...\",\n content: {\n message: \"Hello again!\"\n }\n }\n ]\n },\n {\n id_cert: \"QLAxiohs79034sjkldfny8eppqxncp7n4g9vozeyuiwofxn...\",\n messages: [\n {\n ...\n }\n ]\n }\n]\n
The concrete values held by a message batch are up to the concrete implementation. The prior JSON array depicting an array of message batches is only an example. However, it is mandatory that a message batch holds the following information:
- The ID-Cert used to sign the messages in the batch
- An array of messages, which must at least contain the following information:
- The signature of the message
- The full content of the message
Returning re-signed messages to the server is done in the same format as the server sends them to the client.
"},{"location":"Protocol%20Specifications/core/#722-server-imposed-limits","title":"7.2.2 Server imposed limits","text":""},{"location":"Protocol%20Specifications/core/#7221-body-size","title":"7.2.2.1 Body size","text":"
Servers can limit the size of an HTTP request body containing re-signed messages. If a body size limit is imposed, the server must communicate this to clients in their response to a query for messages, which have not yet been re-signed. Communicating the body size limit is done by adding a X-P2-Return-Body-Size-Limit header to the response. If this header is not present or has a value of -1, clients should assume that there is no body-size limit.
"},{"location":"Protocol%20Specifications/core/#7222-interval-between-re-signing-batches","title":"7.2.2.2 Interval between re-signing batches","text":"
Servers can define an interval, which a client must wait for before sending a new batch of re-signed messages to the server.
The server communicates this interval to the client as a response to receiving a batch of re-signed messages from the client. The interval is communicated by adding a X-P2-Wait-Until header to the response. The value of this header is a 64-bit integer. The integer represents a UNIX timestamp, which in turn represents the time, at which the client is allowed to send the next batch of re-signed messages.
Clients should expect that the duration of the interval changes between batches. The server can dynamically adjust the duration, which a client must wait before being allowed to send the next batch of re-signed messages. The server can also select to not impose an interval between re-signing batches. Clients should also expect that the server suddenly decides to impose an interval between re-signing batches, even if it has not done so before.
If this header is not present or has a value of -1, clients should assume that there is no interval between re-signing batches.
Fig. 7: Sequence diagram depicting the re-signing procedure.
"},{"location":"Protocol%20Specifications/core/#73-moving-data","title":"7.3 Moving data","text":"
In cases of an imminent server shutdown or distrust in the old server, moving data from the old server is necessary to prevent data loss. This process extends upon the reassigning ownership process, and usually involves the following steps:
- Using the old account, the client requests a data export from your old home server.
- The old home server sends a data export to the client. The client will check the signatures on the exported data, to ensure it was not tampered with.
- The new account re-signs the data with its own keys and imports it into the new home server.
- The new home server verifies the data and signals that the import was successful.
- The old client requests the deactivation or deletion of the old account on the old home server.
sequenceDiagram\nautonumber\n\nparticipant sa as Server A\nparticipant sb as Server B\nbox Same Device\nactor aa as Alice A\nactor ab as Alice B\nend \n\naa->>sa: Request data export\nsa->>aa: Data export\naa->ab: Data shared on device\nab->>ab: Verify data integrity\nab->>ab: Re-sign data\nab->>sb: Request data import\nsb->>sb: Verify data integrity\nsb->>ab: Data import successful\naa-xsa: Deactivate account
Fig. 8: Sequence diagram depicting the data moving process.
How this process is implemented is up to P2 extensions to define. The above steps are only a guideline. The API routes for data export and import are documented in the API documentation
"},{"location":"Protocol%20Specifications/core/#731-content-addressing-with-relative-roots","title":"7.3.1 Content Addressing with relative roots","text":"
Moving data from one server to another might break references to this data. To prevent this as much as possible, content addressing with relative roots is recommended for data behind an additional layer of indirection.
Example
In a chat service, a user might have posted a message containing a picture. In this example, the picture is stored on the user's home server, which is not necessarily the same server as the chat service. If the user moves their account to another server, the picture might not be accessible anymore.
Content addressing with relative roots aids in preventing this issue. Instead of referring to the absolute URL of the content, the server processing the content generates a unique identifier. This identifier can be used to retrieve the content from the server. Most importantly, this identifier does not change when the content is moved to another server. If the base domain of the new server is known, the identifier can be used to retrieve the content from the new server. The \"relative root\" is the base domain of the server, which is used to retrieve the content.
The uniqueness constraint of the identifier is important. If a collision occurs when trying to move the content to another server, the content cannot be migrated in a way that preserves the references to it. One way to ensure the uniqueness of the identifier is to use a hash function on the content itself. Combining this has with a cryptographically strong nonce, then hashing the result of concatenating the nonce and the hash of the content should yield a unique identifier.
The API route for content addressing with relative roots is formatted as follows:
<server_url>/.p2/core/content/<content_id>
The API route for content addressing with relative roots is documented more thoroughly in the API documentation.
Servers with no need for content addressing with relative roots can select to not implement this feature. Servers not implementing this feature should return a 404 Not Found status code when the API route is accessed. Clients should expect finding servers not implementing this feature.
"},{"location":"Protocol%20Specifications/core/#74-challenges-and-trust","title":"7.4 Challenges and trust","text":"
Changing the publicly visible ownership of actor data requires the chain of trust to be maintained. If an \"old\" account wants to change the publicly visible ownership of its data, the \"old\" account must prove that it possesses the private keys that were used to sign the messages. This is done by signing a challenge string with the private keys. If the server verifies the challenge, it authorizes the new account to re-sign the old account's messages signed with the verified key. Instead of overwriting the message, a new message variant with the new signature is created, preserving the old message.
Implementations and protocol extensions should carefully consider the extent of messages that can be re-signed.
Example
In the case of a social media platform with quote-posting functionality, it is reasonable to assume that re-signing a quoted post is allowed. However, this would likely change the signature of the quoted post, which would be undesirable. Edge cases like these are up to implementations to handle, and should be well documented.
"},{"location":"Protocol%20Specifications/core/#8-protocol-extensions-p2-extensions","title":"8. Protocol extensions (P2 extensions)","text":"
polyproto leaves room for extensions, outsourcing concepts such as concrete message types to protocol extensions. This allows for a more flexible core protocol, which can be adapted to a wide variety of use cases. The following sections define:
- protocol extensions, also called P2 extensions
- how protocol extensions interact with the core protocol
- requirements, which must be fulfilled by protocol extensions to become officially endorsed
"},{"location":"Protocol%20Specifications/core/#81-extension-design","title":"8.1 Extension design","text":"
P2 extensions should be either of the following:
- a major technological addition, which can be taken advantage of by other extensions. Examples of this are:
- a unified WebSocket Gateway connection scheme
- Message Layer Encryption (MLS)
- Compatibility with other protocols (e.g. Matrix, ActivityPub)
- a definition of a service. Examples of this are:
- A federated chat application
- A federated social media platform
A good P2 extension should never be both at the same time. If a P2 extension is both a major technological addition and a document describing a particular application use case, it should likely be split into two separate extensions.
Designing P2 extensions, which only specify a single route or a small set of behavior changes is discouraged. Instead, these should be implemented as part of a larger extension, which offers a more comprehensive set of features.
Note
If you are, say, developing a polyproto server implementation with a feature that is not part of the default polyproto specification, you do not have to create a P2 extension for this feature. P2 extensions are useful for defining interoperable services, which can be implemented by a variety of servers and clients.
"},{"location":"Protocol%20Specifications/core/#82-namespaces","title":"8.2 Namespaces","text":"
A namespace is a string used to identify a specific P2 extension. Used as a prefix in URLs, they prevent route name collisions between different extensions. Namespaces should be unique and descriptive. They must only contain lowercase letters, numbers, hyphens, and underscores. Namespaces must be at least 2 characters long and at most 64 characters long.
Officially endorsed P2 extensions have priority over selecting namespaces. If a namespace is already taken by an officially endorsed extension, a different namespace must be chosen. If a namespace collision exists between an officially endorsed extension and a regular P2 extension, the officially endorsed extension has priority.
"},{"location":"Protocol%20Specifications/core/#83-officially-endorsed-extensions","title":"8.3 Officially endorsed extensions","text":"
Officially endorsed extensions are extensions that either:
- have undergone review and approval by the polyproto maintainers
- have been developed by the maintainers themselves
- have been developed by a third party and are now maintained by the polyproto maintainers
Contact the polyphony-chat maintainers under info@polyphony.chat if you want to have your extension officially endorsed.
Officially endorsed extensions must fulfill all the requirements listed in section 8.
Each version of an extension developed by outside parties must undergo the review process before being officially endorsed.
"},{"location":"Protocol%20Specifications/core/#84-versioning-and-yanking","title":"8.4 Versioning and yanking","text":"
Semantic Versioning v2.0.0 is used for versioning P2 extensions. The version number of an extension is defined in the extension's documentation. The version number must be updated whenever a change is made to the extension. The only exception to this rule is when marking an extension as deprecated (yanking).
"},{"location":"Protocol%20Specifications/core/#841-yanking","title":"8.4.1 Yanking","text":"
Yanking an extension means that the extension is no longer supported, and that it should not be used. A later version of the extension should be used instead. Yanked extension versions should prominently display the \"yanked\" status next to the version number in the extension's documentation.
Versions of officially endorsed P2 extensions can normally not be removed, only marked as yanked.
"},{"location":"Protocol%20Specifications/core/#85-dependencies","title":"8.5 Dependencies","text":"
P2 extensions can depend on other P2 extensions. If an extension depends on another extension, the name of the dependency must be listed in the extension's documentation, along with a link to the dependencies' specification document.
The following syntax is used for indicating the version number of a dependency:
Syntax Meaning
1.0.0 Any version of the dependency with the major version
1, a minor version of
0, and a patch version of
0 or greater is required.
1.0 Any version of the dependency with the major version
1 and the minor version
0 is required. The patch version is unimportant.
1 Any version of the dependency with the major version
1 is required. The minor and patch versions are unimportant.
When selecting a version number for a dependency, the highest possible version number that fulfills the requirements should be selected.
The name of the dependency along with the version number is to be listed right beneath the extension's version declaration in the extension's documentation. Ideally, a link to the dependencies' specification document should be included.
To grow the ecosystem of interoperable services, it is encouraged to first develop a generic version of that service, which acts as a shared base for all implementations. This shared base can then be extended with the exact, non-service-critical features that are needed for a specific implementation.
For example, a generic, federated chat service extension might offer routes for adding reactions to chat messages. However, a route for adding reactions with full-screen animation effects would be better suited as an implementation-specific detail.
If possible for the given use case, P2 extensions should depend on and extend already existing, officially endorsed P2 extensions.
Example
Say, you are developing a social chat platform using polyproto. In this example, you would like your chat platform to have a feature, which is not part of the officially endorsed polyproto-chat extension. Instead of developing a new extension from scratch, your chat extension should likely depend on polyproto-chat, and define only this new feature as part of your own extension.
Doing this ensures a high level of interoperability across all different implementations of a specific application group.
"},{"location":"Protocol%20Specifications/core/#86-routes","title":"8.6 Routes","text":"
Polyproto extensions must never change, add or remove routes defined by the extension they depend on. Instead, routes with alternating or new behavior must be added under a newly defined namespace, which must differ from the original namespace. Changing the behavior of existing routes breaks compatibility with other implementations of the same extension.
Route paths must always start with .p2/, followed by the extensions' namespace. Namespaces are explained in section 8.2.
"},{"location":"Protocol%20Specifications/core/#9-services","title":"9. Services","text":"
Info
A \"service\" is any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services.
Actors can use their identity to register with any server hosting polyproto services, such as polyproto-chat. These servers can be the actors' home server, but can also be foreign servers. There is no limitation to how many services any given actor can register with, and what these services are.
Application specific implementations of polyproto should consider that users of their service might also want to register for services offered by other servers, using the same identity.
"},{"location":"Protocol%20Specifications/core/#91-discoverability","title":"9.1 Discoverability","text":"
The discoverability feature allows users who are registered with the same service but on different servers to communicate with each other. The actor initiating the communication only needs to know the federation ID of the actor they want to communicate with. Consider the following example:
Example: Discovering services
Info
The example below is simplified for the sake of clarity. In a real-world scenario, Alice and the Chat server would perform the foreign server authentication procedure described in section 4.1.1 before Alice can send a chat message to Bob. The example also uses a simplified example of how polyproto-chat works.
Alice and Bob want to communicate with each other. Both Alice and Bob are registered on servers which host the polyproto-chat service. However, Alice and Bob are not registered on the same server, and they do not share any chat rooms. Alice types in Bob's federation ID into her chat client. The client then queries Bob's home server to find out, which server Bob is using for the polyproto-chat service. Alice's client can then send the chat message to Bob's server, which will forward the chat message to Bob.
sequenceDiagram\nautonumber\n\nparticipant sb as Bob's Home Server\nactor aa as Alice\nparticipant sc as Chat server Bob is registered on\nactor ab as Bob\n\naa->>sb: Query: Which server is Bob using for polyproto-chat?\nsb->>aa: Response: URL of Chat server Bob is registered on\naa->>sc: Message to Bob\nsc->>ab: Forward message from Alice to Bob
Fig. 9: Sequence diagram depicting how Alice's client discovers which server Bob is using for the examplary polyproto-chat service.
The example demonstrates how Alice can communicate with Bob, even though they do not share any servers.
To be discoverable, an actor must add a key-value pair to their home server's database. The key is the name of the service, and the value is the base URL of the server hosting the service.
The API routes for managing discoverability are documented in the API documentation
"},{"location":"Protocol%20Specifications/core/#911-changing-a-primary-service-provider","title":"9.1.1 Changing a primary service provider","text":"
Keys are unique in the actor-scoped service->service-provider table. Actors wanting to register for two or more different implementations of the same service must select, which service provider to use as a so-called \"primary service provider\" for that service.
If the actor is human, clients must not override the existing key-value pair silently. Instead, clients must either ask the actor to confirm the change, or not change the key-value pair. Automated actors may override values as they see fit.
Changing a primary service provider entry is considered a sensitive action and should require a second factor of authentication.
Messages do not get moved or re-signed when changing the primary service provider for a given service. If an actor wants to move their messages to the new primary service provider, they must request a migration.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/","title":"P2 Extension: polyproto-auth","text":"
v1.0.0-alpha.1 - Treat this as an unfinished draft. Semantic versioning v2.0.0 is used to version this specification. The version number specified here also applies to the API documentation.
The polyproto-auth extension is a protocol extension for polyproto that provides a basic authentication mechanism to register new users and authenticate existing users.
- P2 Extension: polyproto-auth
- 1. Registration of a new actor
- 1.1 Registering a new actor on a polyproto home server
- 1.2 Authenticating a new client on a polyproto home server
- 1.3 Authenticating on a foreign server
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#1-registration-of-a-new-actor","title":"1. Registration of a new actor","text":"
Registering a new actor in the context of polyproto is done through an API route defined in the polyproto-auth \"No registration needed\" API documentation.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#11-registering-a-new-actor-on-a-polyproto-home-server","title":"1.1 Registering a new actor on a polyproto home server","text":"
To register, the client sends the necessary information to their home server. The server verifies the data, checks username availability, and responds with HTTP 201 and the new identity's federation ID, if successful. However, a session token is not provided until the actor authenticates a client, as detailed in section 1.2.
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant s as Server\n\nc->>s: Registration information\ns->>s: Verify correctness of provided information,<br />check if username is available, etc\n\nalt verification successful\n s->>s: Verify provided CSR\n\n alt CSR okay\n s->>s: Sign CSR\n s->>c: HTTP status code 201, with actor federation ID\n end\nend
Fig. 1: Sequence diagram of a successful identity creation process.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#12-authenticating-a-new-client-on-a-polyproto-home-server","title":"1.2 Authenticating a new client on a polyproto home server","text":"
To access their account from a new device, an actor authenticates the session with their home server by sending authentication information and a certificate signing request (CSR) for the new client. If verified successfully, the server signs the CSR and responds with the newly generated ID-Cert and a session token corresponding to this ID-Cert.
sequenceDiagram\nautonumber\n\nactor c as Client\nparticipant s as Server\n\nc->>s: Auth information, CSR\ns->>s: Verify correctness of provided auth information\n\nalt Verified successfully\n s->>s: Verify provided CSR\n alt CSR okay\n s->>s: Sign CSR\n s->>c: HTTP status code 201, ID-Cert + session token\n end\nend\n
Fig. 2: Sequence diagram of a successful client authentication process.
The client is now authenticated and can use the session token and ID-Cert to perform actions on behalf of the actor identified by the ID-Cert.
"},{"location":"Protocol%20Specifications/P2%20Extensions/auth/#13-authenticating-on-a-foreign-server","title":"1.3 Authenticating on a foreign server","text":"
Authenticating on a foreign server requires the actor to sign a challenge string with their private identity key and send it, along with their ID-Cert, to the server. The server then validates the ID-Cert's origin, the challenge string's signature, and the ID-Cert's validity.
If the verification is successful, the foreign server can issue a session token to the actor.
Example: Say that Alice is on server A, and wants to authenticate on Server B, using her existing identity.
Alice's client sends a request to Server B for a challenge string, telling Server B the session ID they are communicating from in the process. Upon receiving a response, Alice signs this challenge string with the correct private key. They then send the signature to Server B. Server B can now verify that it was actually Alice who signed the string, and not a malicious outsider. Server B does this by requesting Alice's ID-Cert, specifically the ID-Cert matching the session ID Alice identified with to Server B. If all goes well, server B will send a newly generated session token back to Alice's client. Alice's client can then authenticate with server B by using this token.
sequenceDiagram\nautonumber\n\nactor a as Alice\nparticipant sb as Server B\nparticipant sa as Server A\n\na->>sb: Challenge string request including current Session ID\nsb->>a: Challenge string\na->>sb: Signed challenge, ID-Cert, optional payload\nsb->>sa: Get Server A Public Certificate\nsa->>sb: Send Public Certificate\nsb->>sb: Verify signature of challenge string\nsb->>a: Session token, optional payload
Fig. 3: Sequence diagram of a successful identity verification.
In the diagram, Alice's \"optional payload\" is extra data that might be requested by servers. This is useful when using a single identity across various polyproto implementations, due to differing information needs. The payload is signed with the actor's private identity key.
Likewise, the \"optional payload\" sent by the server in the above diagram can be used by implementations to send additional information to the client. An example might be initial account information.
Example
Alice currently has a polyproto identity, which she created when signing up for \"https://example.com/chat\". When signing up for this service, she didn't need to provide any additional information on registration. However, when she wants to actor her existing identity to sign up for \"https://example.com/social\", she is asked to provide her email address, which she can provide as the \"optional payload\". The server can then store the email address in its' database, associate it with Alice's identity, and let Alice log in with her existing identity.
If Alice's session token expires, they can repeat this process of requesting a challenge string and, together with her ID-Cert, exchange it for a session token. However, if Alice wants to access this third party account from a completely new device, they will have to perform the steps described in section 1.2 to obtain a valid ID-Cert for that session.
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/","title":"P2 Extension: polyproto-chat","text":"
TODO
TODO: This is a work in progress. Chat-related content is currently being migrated over from the polyproto-core specification. This document is not at all complete.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#4-federating-directgroup-messages","title":"4. Federating direct/group messages","text":""},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#41-direct-messages","title":"4.1 Direct messages","text":"
Federating direct messages is straightforward. When Alice sends a message to Bob, their client will send the message to Bob's home server via an API request. Bob's home server will then send the message to Bob's client via an established WebSocket connection, and vice versa.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#42-group-messages","title":"4.2 Group messages","text":"
Group messages work just like guilds, in the sense that data is stored by the home server of the group's creator, meaning that all group members will have to communicate with the group creator's home server. If the group creator leaves the group, the ownership of the group is transferred to another member. The group chat stays on the group creator's home server.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#6-encrypted-channels-and-groups","title":"6. Encrypted channels and groups","text":"
Note, that in the below sequence diagrams, the MLS Welcome message and the MLS Group notify message are all encrypted using the identity key of the recipient.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#61-encrypted-guild-channels","title":"6.1 Encrypted guild channels","text":"
Encrypting a guild channel is done by a client with the MANAGE_CHANNEL permission. Upon successfully requesting enabling encryption of a channel, all future messages in it will be encrypted. Joining an encrypted channel is done by sending a join request to the server. The server will then notify the channels' members of the join request. The members will then decide whether to accept or reject the join request. If the join request is accepted by any member, that member will initiate the MLS welcoming process. If the member finds that the join request is invalid (perhaps due to an invalid KeyPackage), the join request must be denied. It is imperative that join requests are verified correctly by the server.
Text Only
Charlie Server Alice Bob\n | | | |\n | Channel join request + KeyPackage | | |\n |--------------------------------------------->| | |\n | | | |\n | | Notify group of join request | |\n | |----------------------------------- | |\n | | | | |\n | |<---------------------------------- | |\n | | | |\n | | Channel join request + Charlie's KeyPackage | |\n | |------------------------------------------------>| |\n | | | |\n | | | Verify Charlie's KeyPackage |\n | | |------------------------ |\n | | | | |\n | | |<----------------------- |\n | | | |\n | | Notify group of new member: Charlie | |\n | |<------------------------------------------------| |\n | | | |\n | | Encrypted MLS Welcome | |\n | |<------------------------------------------------| |\n | | | |\n | | Forward: Notify group of new member: Charlie | |\n | |------------------------------------------------------------------------------>|\n | | | |\n | Forward: Notify group of new member: Charlie | | |\n |<---------------------------------------------| | |\n | | | |\n | Forward: encrypted MLS Welcome | | |\n |<---------------------------------------------| | |\n | | | |\n
Fig. 3: Sequence diagram of a successful encrypted channel join in which Alice acts as a gatekeeper. The sequence diagram assumes that Alice can verify Charlies' public key to indeed belong to Charlie, and that Alice accepts the join request."},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#62-encrypted-direct-messages","title":"6.2 Encrypted direct messages","text":"
Adding another person to a direct message is not possible, and would not make much sense, as the new person cannot see any messages that were sent before they joined the group. If Alice wants to add Charlie to a direct message with Bob, she will have to create a new direct message with Bob and Charlie.
Text Only
Alice Server Bob\n| | |\n| Request Bob's KeyPackages | |\n|--------------------------------------------->| |\n| | |\n| Bob's KeyPackages | |\n|<---------------------------------------------| |\n| | |\n| Verify Bob's KeyPackages | |\n| ----------------------- | |\n| | | |\n|<----------------------- | |\n| | |\n| Notify group of new member: Bob | |\n|--------------------------------------------->| |\n| | |\n| Encrypted MLS Welcome | |\n|--------------------------------------------->| |\n| | |\n| | Forward: New group member: Bob |\n| |--------------------------------->|\n| | |\n| | Forward encrypted MLS Welcome |\n| |--------------------------------->|\n| | |\n
Fig. 4: Sequence diagram of a successful encrypted direct message creation.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#63-encrypted-group-messages","title":"6.3 Encrypted group messages","text":"
Encrypted group messages work by using the traditional MLS protocol, with the additional concept of group owners. Only group owners can add new members to the group and forcibly remove others from the group. The Group owner is determined by the Client-Server API.
Text Only
Alice (gatekeeper) Server Bob Charlie\n| | | |\n| Request Bob's KeyPackages | | |\n|------------------------------------------------->| | |\n| | | |\n| Bob's KeyPackages | | |\n|<-------------------------------------------------| | |\n| | | |\n| Verify Bob's KeyPackages | | |\n|------------------------ | | |\n| | | | |\n|<----------------------- | | |\n| | | |\n| Notify group of new member: Bob | | |\n|------------------------------------------------->| | |\n| | | |\n| Encrypted MLS Welcome | | |\n|------------------------------------------------->| | |\n| | | |\n| | Forward: New group member: Bob | |\n| |-------------------------------------->| |\n| | | |\n| | Forward encrypted MLS Welcome | |\n| |-------------------------------------->| |\n| | | |\n| Request Charlie's KeyPackages | | |\n|------------------------------------------------->| | |\n| | | |\n| Charlie's KeyPackages | | |\n|<-------------------------------------------------| | |\n| | | |\n| Verify Charlie's KeyPackages | | |\n|---------------------------- | | |\n| | | | |\n|<--------------------------- | | |\n| | | |\n| Notify group of new member: Charlie | | |\n|------------------------------------------------->| | |\n| | | |\n| Encrypted MLS Welcome | | |\n|------------------------------------------------->| | |\n| | | |\n| | Forward: New group member: Charlie | |\n| |-------------------------------------->| |\n| | | |\n| | Forward: New group member: Charlie | |\n| |------------------------------------------------>|\n| | | |\n| | Forward encrypted MLS Welcome | |\n| |------------------------------------------------>|\n| | | |\n
Fig. 5: Sequence diagram of a successful encrypted group creation with 3 members.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#64-joining-new-devices-from-existing-users","title":"6.4 Joining new devices from existing users","text":"
Regardless of channel or group permissions, a user join request from a new device should be accepted by default.
"},{"location":"Protocol%20Specifications/P2%20Extensions/chat/#65-best-practices","title":"6.5 Best practices","text":"
- In case of encrypted guild channel join requests, it may be a good idea to treat multiple join requests from the same user with different clients as a single join request, when it comes to UI/UX.
- Joining an encrypted channel, even from an already established member with a new device, should be an event distinctly visible to all members of the channel. This is to prevent a malicious user from joining a channel without the other members noticing.
- Actor - An entity represented by a federation ID, registered on a home server. Actors can be users, bots, or any other entity with a federation ID.
- CA, Certificate Authority - Any home server that issues and publicly attests to the validity of ID-Certs. In polyproto, only home servers are CAs.
- Client - Any application used by an actor to connect to an instance.
- CSR, Certificate Signing Request - A request sent to a CA to obtain a certificate. It holds information about the entity requesting the certificate, including their public identity key.
- DN, Distinguished Name - A set of RDNs (Relative Distinguished Names) that uniquely identify a certificate. See https://ldap.com/ldap-dns-and-rdns/
- Federation ID - A unique identifier; In public contexts, usually actor@subdomain.example.com, where bold parts are required and non-bold parts are optional.
- Foreign server - An instance that an actor is not registered on; essentially a third party.
- Home server - The instance that an actor is registered on. Any polyproto-core compliant server hosted on the same domain is also considered a home server. A home server is the instance that publicly attests to the validity of all legitimate ID-Certs issued under its FQDN. A domain can have many home servers, but only one per subdomain.
- ID-CSR - A certificate signing request for a client's identity key pair. It is used to obtain an ID-Cert.
- Identity - Synonymous with \"Federation ID\".
- Identity Key Pair - A key pair associating an identity with a set of cryptographic keys used to sign and possibly encrypt messages.
- Instance - A server hosting polyproto compliant software for clients.
-
Message, Messages: In the context of this protocol specification, a message is any piece of data sent by a client that is intended to be identifiable as being sent by a specific actor. To qualify as a \"message\", this piece of data must also, at any point in time, and also if only briefly, be visible to other users or to the unauthenticated public. Examples of things that would qualify as messages include:
- A message sent to another actor in a chat application
- A post on a social media platform
- A \"like\" interaction on a social media platform
- Reaction emojis in Discord-like chat applications
- Group join or leave messages
- Reporting a post or actor, if the report is not anonymous
-
P2 - Shortened form of polyproto.
- P2 Extension - A polyproto extension.
- polyproto-chat - The chat-API used by Polyphony. An extension of the polyproto protocol, defining the routes and capabilities of the chat-API used by Polyphony.
- polyproto - The core federation protocol and APIs of polyproto, enabling identification and authorization on foreign servers. It is independent of the chat-API used.
- Root Certificate - A certificate used to sign other certificates, establishing a chain of trust. In polyproto, only home servers have root certificates.
- Service: Any application-specific implementation of polyproto, defined by a P2 extension. All services are P2 extensions, but not all P2 extensions are services. polyproto-chat is an example of a service.
- Session - A specific period of authenticated interaction between a client and an instance. During the lifetime of a session, the client can perform actions as the actor they are authenticated as.
- Session ID - See polyproto specification: Section 6.1.1.3
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/","title":"P2 Extension: polyproto-mls","text":"
TODO
This is a work in progress. MLS-related content is currently being migrated over from the polyproto-core specification. This document is not yet complete.
v0.1.0-alpha.1 - Treat this as an unfinished draft. Semantic versioning v2.0.0 is used to version this specification. The version number specified here also applies to the API documentation.
- P2 Extension: polyproto-mls
- 1. Encryption
- 1.1. KeyPackages
- 1.1.1 Last resort KeyPackages
- 1.2 Initial authentication
- 1.3 Multi-device support
The following sections describe the additional behavior that polyproto implementations must implement to support encryption via the Messaging Layer Security (MLS) protocol.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#1-encryption","title":"1. Encryption","text":"
About MLS
Polyproto offers end-to-end encryption for messages via Message Layer Security (MLS). polyproto compliant servers take on the role of both an Authentication Service and a Delivery Service in the context of MLS.
MLS is a cryptographic protocol that provides confidentiality, integrity, and authenticity guarantees for group messaging applications. It builds on top of the Double Ratchet Algorithm and X3DH to provide these security guarantees.
Implementations of polyproto can opt to support encryption to secure communication channels. The selected security protocol for all polyproto implementations is the Messaging Layer Security protocol, given its feasibility within the implementation context. MLS inherently supports negotiation of protocol versions, cipher suites, extensions, credential types, and extra proposal types. For two implementations of polyproto to be compatible with each other in the context of encryption, they must have overlapping capabilities in these areas.
The following sections explain the additional behavior that polyproto implementations utilizing MLS must implement.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#11-keypackages","title":"1.1. KeyPackages","text":"
Warning
The sections 1.1 and 1.1.1 are not exhaustive and do not cover all aspects of MLS and KeyPackages. They exist solely to give a general overview of how KeyPackages are used in polyproto. Please read and understand the MLS specification (RFC9420) to implement polyproto correctly.
A polyproto compliant server must store KeyPackages for all clients registered on it. The KeyPackage is a JSON object that contains the following information:
JSON
{\n \"protocol_version\": \"<Version>\",\n \"cipher_suite\": \"<CipherSuite>\",\n \"init_key\": \"<HPKEPublicKey>\",\n \"leaf_node\": \"<LeafNode>\",\n \"extensions\": \"<Extensions>\",\n}\n
protocol_version denotes the MLS protocol version.cipher_suite indicates the used cipher suite for this KeyPackage. Note that a server can store many KeyPackages for a single actor, to support various cipher suites.init_key is a public key for encrypting initial group secrets.leaf_node is a signed LeafNodeTBS struct as defined in section 7.2. Leaf Node Contents in RFC9420. A LeafNode has information representing a users' identity, in the form of the users' ID-Cert for a given session or client. The LeafNodeTBS is signed by using the actor's private identity key.extensions can be used to add additional information to the protocol, as defined in section 13. Extensibility in RFC9420.
A KeyPackage is supposed to be used only once. Servers must ensure the following things:
- That any KeyPackage is not given out to clients more than once.
- That the
init_key values of all KeyPackages are unique, as the init_key is what makes the KeyPackage one-time use. - That the contents of the
LeafNode and the init_key were signed by the actor who submitted the KeyPackage.
Because KeyPackages are supposed to be used only once, servers should retain multiple valid KeyPackages for each actor, alerting clients when their stock is running low. Consult the \"Registration needed\"-API for more information about how servers should request new KeyPackages from clients. Servers should delete KeyPackages when their validity lapses.
Servers only store KeyPackages for home server users, not for foreign users.
About keys
It is recommended that keys are generated using the EdDSA signature scheme, however, other signature schemes may be used as well. Consider, that intercompatibility can only be guaranteed if all communicating parties have an overlapping set of supported signature schemes.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#111-last-resort-keypackages","title":"1.1.1 Last resort KeyPackages","text":"
A \"last resort\" KeyPackage, which, contrasting regular KeyPackages, is reusable, is issued when a server runs out of regular KeyPackages for an actor. This is to prevent DoS attacks, where malicious clients deplete all KeyPackages for a given actor, blocking that actor's inclusion into encrypted groups or guild channels.
Servers are to replace a \"last resort\" KeyPackage after it has been used at least once by requesting one from the client.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#12-initial-authentication","title":"1.2 Initial authentication","text":"
During the initial authentication process, a client must provide at least one KeyPackage and one \"last resort\" KeyPackage to the server, in addition to the required registration information.
The public identity key inside the LeafNode of this KeyPackage corresponds to the public identity key found inside a clients' ID-Cert.
"},{"location":"Protocol%20Specifications/P2%20Extensions/mls/#13-multi-device-support","title":"1.3 Multi-device support","text":"
polyproto servers and clients employing encryption must support multi-device use. The MLS protocol assigns each device a unique LeafNode and prohibits key sharing across devices. Each device offers distinct KeyPackages and an own ID-Cert.
TODO: Integrate this from the core spec
Text Only
A server identity key's lifetime might come to an early or unexpected end, perhaps due to some sort\nof leak of the corresponding private key. When this happens, the server should generate a new\nidentity key pair and broadcast the\n[`SERVER_KEY_CHANGE`](/docs/APIs/Core/WebSockets/gateway_events.md#server_key_change) and\n[`LOW_KEY_PACKAGES`](/docs/APIs/Core/WebSockets/gateway_events.md#low_key_packages) gateway events\nto all clients. Clients must request new ID-Certs (through a CSR), and respond appropriately to the\n[`LOW_KEY_PACKAGES`](/docs/APIs/Core/WebSockets/gateway_events.md#low_key_packages)\nevent. Should a client be offline at the time of the key change, it must be informed of the change\nupon reconnection.\n\n!!! note\n\n A `LOW_KEY_PACKAGES` event is only sent by servers which use MLS encryption. Server/Clients not\n implementing MLS encryption can safely ignore this event.\n
"},{"location":"Type%20Definitions/","title":"meow","text":""},{"location":"Type%20Definitions/auth/","title":"more meow :3","text":""},{"location":"Type%20Definitions/core/","title":"more meow :3","text":""},{"location":"blog/","title":"Blog","text":""},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/","title":"Self-updating structs, moving blog posts to GitHub, and more!","text":"
Introducing self-updating structs, explaining how they work, and what they are good for. Also, moving blog posts to GitHub, and other improvements.
It has been a while since the last update post - 1 month to be precise! I haven't gotten around to writing one of these, mostly because of personal time- and energy constraints. However, now that these resources are finally replenishing again, I figured that it is once again time!
"},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/#moving-blog-posts-to-github","title":"Moving Blog Posts to GitHub","text":"
This is a pretty self-explanatory point. I thought, that opencollective would find more use by me and other polyphony-curious folk, however, this didn't go as planned. Also, opencollective made their Discord embeds really poopy, which is why I am moving all the blog posts over to GitHub.
"},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/#a-big-one-self-updating-structs","title":"A big one: Self-updating structs","text":"
Ideally, you want entities like Channels, Guilds, or Users to react to Gateway events. A Gateway event is basically a message from Spacebar/Discord to you, which says: \"Hey, User x has changed their name to y!\". If you can reflect those changes immediately within your code, you save yourself from having to make a lot of requests and potentially getting rate-limited.
This is exactly what Self-updating structs set out to solve. The first implementation was done by @SpecificProtagonist and me (thank you a lot again, btw) on the 21st of July. However: This implementation, being in its' infancy, has had some design flaws, which to me made pretty clear, that this whole thing needed to be thought through a little better.
The second iteration of these Self-updating structs was finished... today, actually, by me. It saves memory compared to the first iteration by storing unique objects only once, instead of n = how many times they are being referenced-times. While this way of doing things is really efficient, it also has been a pain in the ass to make, which is precisely the reason why this took me so long. I've learned a lot along the way though.
The public API has also gotten a lot better in \"v2\". This is mostly because I am a big believer in writing tests for your code, and through writing what are essentialy real-world-simulation-examples, I noticed how repetitive or stupid some things were, and thus could improve upon them.
Having this whole thing finished is a big relief. This self-updating thing is an essential feature for any Discord/Spacebar compatible library, and I think that we implemented it very nicely.
"},{"location":"blog/2023/08/17/self-updating-structs-moving-blog-posts-to-github-and-more/#documentation-and-other-improvements","title":"Documentation and other improvements","text":"
@kozabrada123 took it upon himself to re-write a lot of the codes' Documentation. Thanks for that! This will massively improve the ease of use of this library - both when developing for and with it. koza also improved our CI/CT pipeline by incorporating build-caching into it, which speeds up builds.
This has been the last month of Polyphony. In the coming weeks, I will be working on - Implementing self-updating-struct behavior for every struct which needs it - Fixing bugs - Adding more features, like emojis, 2FA, Guild Settings, etc.!
See ya next time!
"},{"location":"blog/2023/08/29/chorus-alpha-010/","title":"chorus Alpha 0.1.0","text":"
We are alpha now! As of 2 days ago, the first Alpha of Chorus, Version 0.1.0, has been released for everyone to look at and use on crates.io!
So, is the library complete now? No. And yes! It's, well, complicated... Let me explain!
Chorus is at a point where I can comfortably say that, if you take voice-support out of the calculation for a bit, the foundation feels rock-solid, easy to work with and easily expandable. However, to stay with our house/building metaphor for a bit, the walls aren't painted yet, there's barely any furniture and not all of the electrical outlets have been installed yet.
Okay, enough with this bad metaphor; What I meant to convey is, that a lot of the API endpoints have not yet been implemented, and there are at least a few points we haven't addressed yet - like Gateway Error Handling, to name an example.
But for an early Alpha, this, in my opinion, is absolutely acceptable. Implementing API endpoints is something that probably someone who is entirely new to Rust could do, given that we've streamlined the procedure so much, and the other stuff can comfortably be fixed without having to do any major changes to the internals.
I, for one, am currently experimenting around with the Polyphony Client, which, by the way, will likely be written with Iced as a GUI Framework, not GTK. I have no prior experience in GUI/Desktop Application development, but I am feeling more confident than ever and I'm eager to learn all there is to know about these topics.
That's that! Seeya next time. Cheers,
Flori
"},{"location":"blog/2023/09/02/getting-started-with-the-polyphony-client/","title":"Getting started with the Polyphony Client","text":"
Us labeling Chorus to be in a public-alpha state was really great news for me, for a lot of reasons! It marked a point in Polyphonys history where, after all these months of work, we agreed upon the fact that what we have is good enough to be shown to the public, and that's always a nice thing when investing so much of your free-time into a project. The other main reason why this is such a great thing is, because this alpha state (at least to me) means, that the public API is kind-of stable, or at least stable enough so that I, the project lead, can rely upon the fact that all the public methods will not, in fact, be replaced in 4 days.
This means, that I can finally start working on the Client! And I have done that! For the past 2? 3? Days, I've been tinkering around with Iced-rs (a really, really great UI framework for Rust, written in Rust) and the client repository to create the 'skeleton' of the application. While this is definitely not trivial, especially since I have no prior experience in desktop application development, it's also not too hard either.
While Iced is not mature yet, and \"how-to\" guides, as well as the promised Iced-book, are still largely missing, the maintainers have done a great job with providing a LOT of code examples and solid rustdocs. It's a fun library/framework to work with, and the Elm-inspired approach of dividing up State, Messages, View- and Update-Logic feels really intuitive and seems to make sure that your Application will never end up in an unexpected state.
That's all I have for today. Thanks for reading this! Here's a video of multi-user login already working ^^
"},{"location":"blog/2023/11/23/porting-chorus-to-webassembly--client-update/","title":"Porting chorus to WebAssembly + Client Update","text":"
What the current state of GUI libraries in Rust means for Polyphony and chorus, and why we are porting chorus to WebAssembly.
Hi all!
To make this part of the post short: The web-based client will be worked on before the native one, if there even ever will be one. The reason is that no currently available native Rust GUI library meets the standards I'd like to see when using it to build an application I am putting my name behind. I'd like to have - accessibility - great styling - cross compilation - memory safety
and the current state of Rust GUIs essentially tells me to \"pick three\", which is unacceptable to me. A WebAssembly based application is the best we'll get for now, and I am fine with that.
Compiling to WebAssembly isn't all that easy though: The wasm32-unknown-unknown target intentionally makes no assumptions about the environment it is deployed in, and therefore does not provide things like a net or filesystem implementation (amongst other things). Luckily, adding support for this compilation target only took me a full 40h work week [:)], and we are now the first Rust Discord-API library (that I know of) to support this target.
You might not have yet heard much about WebAssembly: In the past, web developers could only really use three languages - HTML, CSS, and JavaScript - to write code that browsers could understand directly. With WebAssembly, developers can write code in many other languages, then use WASM to convert it into a form the browser can run.
This is particularly helpful for programs that require a lot of computing power, like video games or design software. Before, running such programs in a browser would be slow or impossible. WebAssembly can make these run smoothly, right in your web browser.
Overall, WebAssembly is expanding the kinds of applications that can be run on the web, making the web a more flexible and powerful place to work and play. Compiling Chorus for WASM allows us to leverage this fairly new technology and bring all of Rusts benefits into a web context.
The next blog post will likely be about progress with the web-based client. See ya until then! :)
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/","title":"Account migration in polyproto","text":"
Account migration is an important and difficult thing to get right in federated systems. In this blog post, I will outline how I imagine account migration to work in polyproto, and what benefits this approach brings.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#account-migration-in-polyproto","title":"Account migration in polyproto","text":"
It seems that striking a good balance between user experience, convenience and privacy has been a difficult task for many federated systems, when it comes to account migration. polyprotos' approach to how data is distributed and stored, and how identities are managed, makes it possible to have a very smooth and secure account migration process.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#the-problem","title":"The problem","text":"
Using Mastodon as an example; When a user wants to move from one instance to another, they have to create a new account on the new instance, and follow all the people they were following on the old account. All the toots and other data from the old account are left behind, and you do not have a way of porting them over to the new account. This is a problem that has been around for a long time, and it is not just a problem with Mastodon, but with many other federated systems as well.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#how-polyproto-works-briefly","title":"How polyproto works, briefly","text":"
In polyproto, your federation ID, e.g. xenia@example.com, is what identifies you. If you want to use this identity on a client, your client will generate a key pair for a certificate signing request, and send this request to your home server. Given that you didn't provide any invalid data, your home server will sign the certificate, and send it back to you.
Any data you send to anyone - be it a chat message, a social media post, or anything else - is signed using your private key. This signature can be verified by anyone using your public key, which is part of the certificate you received from your home server. To check a certificates' validity, you can ask the home server for its root certificate, and verify the signature on the certificate you received.
This means:
- All the data you send is cryptographically tied to your identity
- Anyone can verify that the data was actually sent by you
- Anyone can verify that the data was not tampered with by anyone else
- Everybody can verify that you are who you say you are
This is even true when you are sending data to a different server than your home server.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#migrating-an-account-on-polyproto","title":"Migrating an account on polyproto","text":""},{"location":"blog/2024/02/07/account-migration-in-polyproto/#low-data-centralization","title":"Low data centralization","text":"
Fundamentally, the process of migrating an account in polyproto relies mostly on changing data ownership, rather than moving data around. This works best in scenarios where data is highly distributed, and not stored in a central location.
Example
This might be the case in a social chat messaging system similar to Discord, where messages are stored on the servers of the people hosting the chat rooms.
When you want to move your account from one server to another, you:
- First, create a new account on the new server
- Then, you configure the new account to back-reference the old account
- Next, if you are able to, you tell your old home server about the move
- Last but not least, you verify to the servers storing your data that you are the same person as the one who created the old account. The servers then update the data ownership to your new account. This is done by using your old private key(s), in a way that does not reveal your private key(s) to anyone else.
If applicable, your friends and followers will also be notified about the move, keeping existing relationships intact.
Note
This entire process does not rely on the old server being online. This means that the process can be completed even if the old server is down, or if the old server is not cooperating with the user.
However, including the homeserver in the process adds to the general user experience. If you, for example, have included your federation ID as part of another, non-polyproto social media profile, the old server can automatically refer people to the new account.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#moving-data","title":"Moving data","text":"
Should data actually need to be moved, for example when the old server is going to be shut down, or if the centralization of data is higher, the migration process is extended by a few steps:
- Using the old account, your client requests a data export from your old home server.
- The old home server sends you a data export. Your client will check the signatures on the exported data, to make sure that the data was not tampered with.
- You then import the data into your new account on the new home server.
"},{"location":"blog/2024/02/07/account-migration-in-polyproto/#conclusion","title":"Conclusion","text":"
polyproto's approach to account migration is very user-friendly, and does not require the user to do anything that is not already part of the normal usage of the system. The process is also very secure, as it relies on the cryptographic properties of X.509 certificates, and also works across a highly distributed data model, which, in my opinion, is how the internet should be.
The biggest drawback to this approach is that there are a whole lot of web requests involved. Depending on the amount of data, this can take some minutes or possibly even hours.
It is also worth noting that all of this does not require any new or young technology. polyproto relies on X.509 certificates, which have been around for a long time, and are widely used in many different applications. This means that the technology is well understood, and that there are already many great tools in all sorts of programming languages available to work with it. From my point of view, there is no need to reinvent the wheel.
I hope that this article has given you a good understanding of how account migration works in polyproto. If you have any questions or feedback, feel free to reach out to me via E-Mail, where I can be reached under flori@polyphony.chat. OpenPGP is supported, and my public key can be found on keys.openpgp.org (click to download pubkey)
"},{"location":"blog/2024/02/19/x509-in-polyproto/","title":"Certificates, please: X.509 in polyproto","text":"
This blog post covers a bit about how and why X.509 is used in polyproto, and how we try to make the process of implementing your own server and incorporating it into an existing network a little easier.
Authors' note
Before knowing and reading about the X.500- and PKCS-series of RFCs, I legitimately thought, that implementing an own certificate standard for polyproto would be a good idea! Looking back, this is incredibly naive. But learning new things and improving myself is one of the biggest joys I experience when writing software, so this humbling experience was totally worth it for me, personally.
polyproto is a federation protocol that uses X.509 Public Key Infrastructure (PKI) to prove and federate your identity across a whole network of decentralized services.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#x509","title":"X.509","text":"
Specifically, polyproto leverages the already well-documented and widely used X.509 standard at its core. X.509 was chosen over OpenPGP because of its comparative simplicity. The Web of Trust from OpenPGP often requires active user input to assign trust levels to users and their keys, which is not inline with our ideas and goals for user experience in a decentralized system. Ideally, decentralization and federation is as seamless as possible for the end-user, and X.509 with its Certificate Authority (CA for short) model is the better fit for such a goal. In fact, X.509 can be so seamless to the end-user, that you have probably forgotten that you are already using it right now!
HTTPS (SSL/TLS) certificates are likely the most popular form of digital certificate out there, and they\u2019re implemented in a way, where the only time us humans ever have to think about them, is when our browser tells us that a certificate from a website we\u2019re trying to visit, is not valid anymore.
This popularity is great news for polyproto, because it means that mature tooling for all sorts of programming languages exists today, along with tutorials and documentation, teaching potential implementers how everything works.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#how-polyproto-uses-x509-briefly","title":"How polyproto uses X.509, briefly","text":"
In polyproto, home servers act as Certificate Authorities, while each client you connect from has its own end-user Certificate, issued by your home server. With certificates, you can prove your identity to any person or server at any time. Certificates are also used to verify the integrity of data sent across the polyproto network.
If servers and clients have well-implemented cryptography, it should be extremely unlikely - if not impossible - for non-quantum-based, non-supercomputer-cluster home servers to alter the contents of a message before passing them on to the recipient.
Authors note
Technically, polyproto and X.509 absolutely support Post-Quantum Hybrid Digital Signatures. If these Hybrid Digital Signatures use well-made Post-Quantum Signature schemes and are implemented well, polyproto also offers post-quantum-computing resilience. There seems to be very little, easy to understand reading material on hybrid schemes out there. The best/most easy to understand definition or explanation of hybrid schemes I could find is this one, in the document \"A Hybrid Signature Method with Strong Non-Separability\".
In short, clients generate a PKCS #10 Certificate Signing Request (CSR). This CSR includes some information about the client. In polyprotos case, this information is:
- session ID
- federation ID
- algorithm used to generate the public key attached to the CSR
- the public key attached to the CSR
- a signature which is verifiable using the attached public key, validating all of the aforementioned information
This CSR is sent to your home server, which verifies this information and in turn responds with a polyproto X.509 Certificate (ID-Cert).
Home servers get their root certificate by self-signing a CSR. Unlike actor/client certificates, the home server root certificate features X.509 extensions such as the \"Basic Constraints\" attribute, marking its certificate as a CA certificate, allowing the home server to sign CSRs using this certificate.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#but-its-not-all-perfect","title":"But it\u2019s not all perfect.","text":"
Root Certificates in the context of HTTPS and the modern, SSL/TLS protected web are a big source of centralization. This centralization might be necessary to a degree, but it inevitably means less plurality, and way more hoops to jump through, should you also want to be a CA.
To give context for those who might need it, essentially, every certificate for every website out there has to be able to be traced back to one of the root certificates installed on your internet-capable device's operating system or web browser. This creates an incredible amount of centralization, because one Root Certificate Authority is directly responsible for hundreds of thousands, if not millions of websites. This dependency on a few privileged Root CAs has been monetized, which is why getting an SSL/TLS certificate for your website used to cost you money (and depending on who you are, it might still be that way). Nowadays though, Let's Encrypt exists, offering free SSL/TLS certificates, with the caveat that these certificates are only valid for three months at a time.
"},{"location":"blog/2024/02/19/x509-in-polyproto/#what-can-we-do-about-this","title":"What can we do about this?","text":"
To try and keep open polyproto networks to stay open for everyone, polyproto should make centralization to the degree of modern-day SSL/TLS at infeasible.
An approach we are taking is limiting the length of the certification path.
In X.509, to validate and trust a certificate, you must also trust all the other certificates leading up to the Root Certificate of the Certificate Tree.
graph LR\n A[Root CA] --> B[CA 1]\n A --> C[CA 2]\n B --> D[Middleman]\n D --> E([Leaf Certificate 1])\n C --> F([Leaf Certificate 2])
Example
To trust Leaf Certificate 1, one would have to also trust the certificates held by the Middleman CA, CA 1 and the Root CA.
This path from the certificate you are actually trying to validate to the Root Certificate is referred to as the certification path. By arbitrarily limiting the length of this path, it becomes harder for one certificate authority to issue and manage a great (1.000.000+) number of certificates, due to the increasing amount of processing power required to handle web requests and to verify and sign CSRs.
In polyproto, the maximum length of this certification path is 1, meaning a Root Certificate may only issue leaf certificates. Cutting out middlemen makes it hard to scale to monstrous levels of centralization, as the control one CA can have over the entire network is limited.
All of these factors combined should always make developing or hosting your own home server a viable option.
Authors note
To clarify, this does not mean that polyproto servers will only be able to handle a small amount of users, or that polyproto is designed for small-userbase scenarios. A well-implemented and fast home server implementation should, with the given resources, be able to handle a great number of registered users. This shallow-depth trust model should aid in stopping trust hierarchies with great amounts of influence over the network from forming.
However, real-life power distribution scenarios can be be unpredictable, which means that the efficacy of limiting the certificate path length as a measure to prevent centralization can only be proven when polyproto is being deployed in the real world.
If you have any questions or feedback, feel free to reach out to me via email, where you can reach me under flori@polyphony.chat. OpenPGP is supported, and my public key can be found on keys.openpgp.org (click to download pubkey)
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/","title":"Work on polyproto and taking a break","text":"
In this little update post I write about what I've done in the last couple of weeks alongside talking about taking just a little break (don't worry, y'all are not getting rid of me!)
It's been more or less two weeks since the last post - time for the next one!
A good amount of commits have been since the X.509 in polyproto was published. Let's break them down a little, shall we?
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#certificate-signing-requests","title":"Certificate Signing Requests","text":"
The polyproto crate can now be used to create very basic - but to the best of my knowledge fully RFC compliant - Certificate Signing Requests! This is cool, because Certificate Signing Requests are how all Actors (Users) in polyproto will request a Certificate from their home server. The generated CSRs can be fully verified using the OpenSSL/LibreSSL CLIs, which is very important, as these two applications are the industry standard when it comes to working with cryptographic standards like X.509.
Specifically, polyproto uses the well-defined PKCS #10 standard to pack up and transport all the needed CSR information to your future home server.
The next steps here are:
- Creating validators for the information supplied in the CSRs
- Implementing methods to create an ID-Cert from a CSR
- Write great documentation for what exactly the data inside of the ID-CSR has to look like to be valid
...and as you might have already guessed, I am already working on all of these things! :) They just take time
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#cleaning-up","title":"Cleaning up","text":"
As fun as designing APIs and software architecture is for me, I don't yet always get all of it right on the first try. This is fine though, as long as you recognize the mistakes you've made, learn from them and clean the mess you've made.
I noticed that, as well-meant as some of the traits and trait bounds I've added, they made implementing polyprotos' base types and traits a lot harder than needed. I've been chipping away at the unnecessary and redundant bits, removing some of these traits entirely.
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#updating-the-specification-document","title":"Updating the specification document","text":"
I really wanted to get started on a reference polyproto implementation before finishing the specification document. This might seem a little counter intuitive, but my thought process was, that implementing the crate in code would force me to think about everything from scratch again, which would make it much easier to spot mistakes I potentially made when writing the specification documentation. These mistakes would primarily be:
- Information that is there, but unimportant
- Information that is important, but not there
- Information that is important, there, but wrong
This turned out to be right. I have added a lot of \"TODO\"s and \"FIXME\"s into the specification document since started working on the polyproto crate. All of these TODOs have since been worked on and removed! This doesn't mean that the specification document is now perfect, but it's already better than before, and it'll only get better as I continue to work on the crate!
Another, notable thing that happened is removing the auth-part from the core polyproto protocol! You might be thinking \"whaaaat? does that mean that there will be no authentication in polyproto??\" but I can assure you, that that's not what this means. Removing the authentication endpoints from the core protocol means that polyproto extensions can now choose authentication technologies and methods for themselves, instead of being forced to implement a bunch of REST-based authentication endpoints they might not even want or use anyways.
I would like to thank @laxla@tech.lgbt for this idea! :> Collaboration and feedback are truly great things, and I am happy to have such a nice group of people on Discord and Matrix who are genuinely interested in the silly thing I/we want to do with Polyphony and polyproto :)
Now for the perhaps biggest and probably most important announcement:
"},{"location":"blog/2024/03/06/work-on-polyproto-and-a-vacation-/#taking-a-little-break-for-my-silly-mental-health","title":"Taking a little break for my silly mental health","text":"
It just dawned on me that March 8th marks the one year anniversary of Polyphony!! That's genuinely so cool, and means that this is the project I have worked on the longest for, out of all of my personal projects.
So yeah - it's been almost a year now! And not a lazy one for me, either.
Content warning
The following paragraph covers the topics of anxiety and depression. If you would not like to read about this, feel free to scroll down until you see a big green box with a check mark. The box indicates that it is safe for you to read again!
Big shocker: I am \ud83d\udc7b\ud83d\udc7b\ud83d\udc7b\ud83d\udc7b depreeeeeeessed \ud83d\udc7b\ud83d\udc7b\ud83d\udc7b\ud83d\udc7b\ud83d\udc7b, and have been for the past... 4-6 years of my life. In that time, I have experienced the absolute lowest points of my life. Luckily, I have the absolute privilege to have a great therapist who I have been with for 2 years now, and I am also on medication which already does a good job (most of the time) at taking the edge off the depression.
As it has been explained to me by my therapist, medication should only be a crutch, though. It should not be the tool you should solely rely on for the rest of your life to deal with extreme (social) anxiety and depression. Other, non-medication-related options should be tried, to potentially get you to stop having to take medication to feel non-completely-absolutely-positively-awful every day.
One of these options is therapy, and, as I've mentioned, I've already been doing that for 2+ years now. It has helped me a great, great deal already, and I can absolutely encourage anyone reading who is feeling similarly to how I've described and who is in the lucky position to get (or at least be put on a waiting list for) therapy, to take the first step. It isn't easy; it can actually feel really really scary at first. But do believe me when I say that a good therapist can absolutely help you to get better.
But one hour of therapy a week can sadly only do so much. This is why I, with the encouragement of my friends, loved ones (particularly my lovely, lovely girlfriend) and my therapist, have decided to admit myself into a mental health clinic that specializes in the treatment of depression, anxiety disorders and the like.
Safety checkpoint reached!
It's now over! :)
Starting on March 20th, I will be leaving my everyday life, my girlfriend, my friends, laptop, work, personal projects and everything else behind to go there, and hopefully leave a good bad part of me behind when I come back.
The clinic is far away though, and leaving absolutely everything behind for a month or possibly a little longer is really, really scary to me. However, I think and hope that the metaphorical plunge into icy water will be worth it for me and my mental health.
When I come back, I'll be better than I was before, which will also mean that I can hopefully be more happy and productive in all aspects of my life, including Polyphony.
If you're reading this on or after March 20th, then see you on the other side :) I hope the grass is greener there!
BEGPOSTING ON MAIN
I am lucky and extremely privileged to have been growing up in Germany, a country with a (mostly) functioning social welfare system and universal health care. If this wasn't the case, I'd likely be absolutely unable to afford to put myself into such good care. Germany doesn't pay for everything though, and the train rides to and from the clinic will likely be expensive for me, as will the 10\u20ac daily fee for staying at a clinic (capped at 280\u20ac).
I can currently afford this without financially ruining myself, so don't worry about that. However, this whole endeavour will take a good chunk out of my current savings. Thus, if you'd like to donate to my ko-fi to help me cover the costs, it would mean a lot to me! <3
Please only do so if you are in a stable financial standing yourself, though. As I said, with or without tips, I'll manage. :)
"},{"location":"blog/2024/06/01/polyproto-extensions/","title":"polyproto extensions.","text":"
polyproto is a new federation protocol. Its main focus is enabling seamless participation of one actor on many different servers. The core specification lacks routes for sending any sort of user generated data anywhere, though. What is up with that?
"},{"location":"blog/2024/06/01/polyproto-extensions/#to-federate-is-to-be-familiar","title":"To federate is to be familiar","text":"
If any application wants to participate in the network of polyproto services, it has to speak the same language as those other services. When wanting to send a message to a server that you are authenticated on, your client needs to know exactly what that HTTP request has to look like. This is nothing new. One take on a solution for this problem stems from the people working on the ATProtocol, who created Lexicon. From the atproto website:
Lexicon TL;DR
Lexicon is a global schema system. It uses reverse-DNS names like \"com.example.ping()\". The definitions are JSON documents, similar to JSON-Schema. It's currently used for HTTP endpoints, event streams, and repo records
The core of polyproto is supposed to be infinitely adaptable, to be flexible enough to be used for just about anything, which is why I do not want to force a fixed set of routes onto every single polyproto implementation.
Lexicon sounds interesting and really versatile! However, as mature as the idea itself might be, it is pretty complex and does not yet seem to have good community support in the form of libraries/crates to aid in working with this new schema system. I also do not want to force polyproto integrations to use a (potentially very complex) Lexicon parser and dynamic routing system thingymajig - although having \"no rules\" means, that if you want to build a polyproto service which uses Lexicon, you absolutely can.
"},{"location":"blog/2024/06/01/polyproto-extensions/#we-need-a-common-foundation","title":"We need a common foundation","text":"
I am a big proponent of defining a set of (mutually independent) protocol extensions, which include additionally needed behavior and concrete HTTP routes for building a specific application. This has the following benefits:
- If you'd like to build a polyproto chat client, and there's a polyproto-chat extension, you simply need to add the additional things required by that extension. No need for complex parsing! Code only what you need and do not care about the rest.
- Mutual independence means being able to combine extensions however you'd like. You could, for example, create a chat app with integrated microblogging functionality.
- Developers are free to come up with whatever they want. How about ActivityPub x polyproto? Since polyproto doesn't define a message format, this is absolutely possible!
- Simplicity! polyproto and its \"official\" extensions will always just have plain old REST APIs, for which tooling is readily available. Why bother with something fancy and dynamic, when this does the trick?
On the other hand, everyone now has to agree on one extension to use for a specific application. You cannot participate on servers, which have use an extension which is completely different from the one that your client implements, as an example.
"},{"location":"blog/2024/06/01/polyproto-extensions/#the-polyproto-foundation-get-it-sigh","title":"...the polyproto foundation. Get it? sigh","text":"
To develop, provide and maintain polyproto and some major \"official\" extensions (such as polyproto-chat), creating a non-profit foundation is likely a good idea for a future where polyproto is actually being used in the real world.
This could sort of be seen like the XMPP Standards Foundation which develops and maintains XMPP extensions. Unlike XMPPs extensions however, official polyproto extensions should always be major additions in functionality. As an example: XEP-0084 is the official XMPP extension for User Avatars. An entire 12 point document, which describes one simple feature!
polyproto extensions should either always be a major technological addition, which can be taken advantage of by other extensions (examples for this would be WebSocket Gateways and Messaging Layer Security), or a document describing a set of routes, which define a particular application use case (A Discord-like, a Reddit-like, a Twitter-like, and so on). Having official extensions adhere to these rules ensures that polyproto will not become a cluttered mess of extensions and that it and its extensions are easy to understand and implement, due to less documentation having to be read and written.
"},{"location":"blog/2024/06/01/polyproto-extensions/#is-this-a-bottleneck-for-me-as-a-developer","title":"Is this a bottleneck for me as a developer","text":"
If you are a developer, you might ask yourself:
Question
Implementing common chat behaviour sounds cool in terms of intercompatibility, but doesn't this limit what I can do with my application? I have planned for a cool feature X to exist in my chat service, but that doesn't exist in the protocol extension!
Extensions should be a usable minimum of common behavior that all implementations targeting the same \"class\" of application must share. Implementations can absolutely offer all the additional special/unique features they'd like, though. polyproto clients implementing the same extensions can be treated as clients with a reduced feature set in this case. What is crucial, however, is that the additional features do not prohibit \"reduced feature set clients\" from using the behavior described in the extension, if any sort of federation or interoperability is wanted.
What works
In your implementation of a chat service, users can send each other messages with special effects, such as fireworks, confetti and similar. A different implementation of polyproto-chat is unlikely to see these special effects on their end. However, they can still see the messages' text contents, send replies to the message, and do all sorts of other things as described in this hypothetical polyproto-chat extension.
What doesn't work
In your implementation of a chat service, users can send each other messages with special effects, such as fireworks, confetti and similar. Your implementation requires every client to send information about the special effect they'd like to send with a message - otherwise sending the message fails. If this is the case and you haven't implemented a sort of \"adapter\" for other polyproto-chat clients, these clients will not be able to send any messages to servers running your chat software. This conflicts with the behaviour required by the polyproto-chat extension and is therefore unacceptable.
Also keep in mind that through clever engineering, it might be possible to write adapters for behavior, which should be required in your implementation and conflicts with the base extension. Picking up the \"What doesn't work\" example again, the implementer could simply \"translate\" message sending requests made to the polyproto-chat endpoints and add the required \"special effects\" information, stating that messages sent through polyproto-chat endpoints have no special effects added to them.
"},{"location":"blog/2024/06/01/polyproto-extensions/#closing-words","title":"Closing words","text":"
I am of the opinion that, while this way of having extensions might not be the most technologically advanced solution, it certainly offers many possibilities while being easy to understand and implement.
These are my current plans, ideas and thoughts for making a v1 of polyproto extensible. If you have any thoughts on this matter, please do let me know! You can contact me via email or by writing a message on our Discord.
Thank you for reading! :>
"},{"location":"blog/2024/06/01/polyproto-extensions/#happy-pride-month","title":"Happy pride month! \ud83c\udff3\ufe0f\u200d\ud83c\udf08\ud83c\udff3\ufe0f\u200d\u26a7\ufe0f\ud83d\udc9b\ud83e\udd0d\ud83d\udc9c\ud83d\udda4","text":""},{"location":"blog/2024/10/14/nlnet-grant-application/","title":"NLnet grant application","text":"
The NLnet foundation is a non-profit organization that supports open-source projects. They have a grant program that funds projects that align with their goals. On behalf of Polyphony and the polyproto project, I have submitted an application for a grant of 10,000\u20ac from the NLnet foundation in their funding round of October 2024.
Should we be successful in our application, the grant will be used to fund the development of the Polyphony and polyproto projects, which would rapidly increase the development velocity of both projects and bring them one big step closer to being ready for a public alpha release.
The application required a bunch of different, interesting questions to be answered. I would like to share the answers with you, as they give a good overview of what we are working on, what we are planning to do, and considerations we have made in the past.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#can-you-explain-the-whole-project-and-its-expected-outcomes","title":"Can you explain the whole project and its expected outcome(s)?","text":"
polyproto is a new federation protocol with the goal of offering account portability and an effortless experience for users and developers alike. It is part of the Polyphony Project, which aims to be a truly competitive alternative to centralized, proprietary chat services like Discord. We came up with the idea for polyproto because of a lingering frustration with current federation protocols and their sub-optimal suitability for such a project.
polyproto is not limited to an application in a chat service, and that it is not incompatible with federation protocols such as ActivityPub. It would technically be possible to write a polyproto + ActivityPub server and client to offer new possibilities to the currently existing Fediverse. We want to empower users, not split userbases further.
Our goal is to deliver the Polyphony chat service with polyproto at its core, build great SDKs for other developers to work with, and to also directly work together with other developers to get alternative implementations of polyproto-based chat services for users to choose from. Polyphony should be the ideal, federated and decentralized Discord replacement; a service, that can be used by teenagers, the elderly and anyone in between and which ideally does not require any additional technical knowledge or proficiency to use.
Documentation/Protocol specification: https://docs.polyphony.chat/Protocol%20Specifications/core/ Simplified overview of the protocol (sadly a little dated, but it can give an overview of the basics nonetheless): https://docs.polyphony.chat/Overviews/core/ API Documentation: https://docs.polyphony.chat/APIs/core/ \"polyproto-rs\" Rust crate: https://github.com/polyphony-chat/polyproto-rs Polyphony organization overview: https://github.com/polyphony-chat \"chorus\" API Wrapper for Discord, Spacebar-Chat (formerly \"Fosscord\") and our own server: https://github.com/polyphony-chat/chorus
"},{"location":"blog/2024/10/14/nlnet-grant-application/#have-you-been-involved-with-projects-or-organisations-relevant-to-this-project-before-and-if-so-can-you-tell-us-a-bit-about-your-contributions","title":"Have you been involved with projects or organisations relevant to this project before? And if so, can you tell us a bit about your contributions?","text":"
I (Flori Weber, bitfl0wer) have been following the Spacebar-Chat (formerly \"Fosscord\") project for some time before deciding to start the Polyphony-Chat GitHub organization in March 2023. The contributions I have made to the Spacebar project were limited to additions and overhauls of the projects documentation, because of a lack of TypeScript knowledge, which is the programming language primarily used in the Spacebar organization. Of course, I have prior experience in software development and software design through my work at Deutsche Telekom MMS GmbH, but this is my first project with such a topic. I am part of a software development community called \"Commune\", which is a home for individuals and groups who also have interest in federated and/or decentralized social media, as well as \"reclaiming\" the web from the hands of large corporations. There, I have access to like-minded individuals who are also very much interested in polyproto and in seeing polyproto succeed.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#requested-amount","title":"Requested Amount","text":"
$10.000
"},{"location":"blog/2024/10/14/nlnet-grant-application/#explain-what-the-requested-budget-will-be-used-for-does-the-project-have-other-funding-sources-both-past-and-present","title":"Explain what the requested budget will be used for? Does the project have other funding sources, both past and present?","text":"
There are three key things that I (Flori Weber, bitfl0wer) have been wanting to tackle for months now, but have never been able to do, because of a lack of available personal time or expertise. These three things are:
- Writing new material and, if necessary, reworking existing material that
- Describes, in a condensed form, what polyproto is about, targeted towards people who do not have [a lot of] existing knowledge in the topics of federation and decentralized social networking concepts ($250-400)
- Describes, in a condensed form, how polyproto works, targeted towards developers who might be interested in learning more about the inner workings of polyproto, without needing to read the entire protocol specification document ($250-400)
- Starting to build some sort of \"brand\" by
- Commissioning an artist to create a recognizable logo for polyproto ($200-400)
- Commissioning a frontend developer to build a landing page for our project, since non-developers do seem to prefer information hosted outside of GitHub and plain looking documentation pages. This landing page would also host the written material mentioned in 1. ($500-1500)
- Paying (freelance) developers to expedite this projects' journey to completion, where there are the following tasks we could use additional brains on:
- Paying developers to start integrating our polyproto crate into our \"chorus\" client library and \"symfonia\" server (~$2000)
- Stabilizing and extending the symfonia server to host a first, publicly usable instance (~$1500)
- Getting additional help from UI/UX designers and frontend developers to build a client mockup, then having this mockup translated into a client prototype which can be hosted alongside the symfonia server (~$2000)
- \"Overhaul\"/Refactoring tasks which we, as a group of mainly university students working part time jobs in addition, simply did not yet have the time to get to (~$1000-1500)
This would total to $7700 or $9700, depending on whether the lower or higher estimate is used. Additionally, I would like to extend the domains polyphony.chat and polyproto.org for some years using the funding and top up our prepaid E-Mail server hosted at https://uberspace.de/en/. The domain and the E-Mail server make up most of our current operating costs, ranging between 7-12$ a month. This might not sound like a lot in the grand scheme of things, but I am currently paying for these out of my own pocket as an undergraduate with little income, so being able to potentially reduce monthly expenses is a nice prospect.
We currently have and have never had additional/other sources of funding. Receiving funding from NLNet would thus be our first source of funding.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#compare-your-own-project-with-existing-or-historical-efforts","title":"Compare your own project with existing or historical efforts.","text":""},{"location":"blog/2024/10/14/nlnet-grant-application/#spacebar-chat","title":"Spacebar Chat","text":"
Already previously mentioned, Spacebar Chat is also working on an open-source Discord replacement in form of offering an API-compatible server. However, they do not seem interested in making Spacebar Chat a federated chat application with good user experience, as their primary focus is to reverse-engineer and re-implement the Discord API spec-for-spec.
From talking to Spacebar Maintainers, their code reportedly seems to have accrued noticeable amounts of technical debt which made it undesirable for most of the maintainers to continue development on the server. Being friends with some of the now mostly inactive maintainers, I have considered forking the server repository to have an already mostly working starting ground to work with. However, due to the reports of technical debt and our organizations' unfamiliarity with JavaScript/TypeScript, we have decided to start from scratch, in Rust.
The accumulated knowledge that Spacebar contributors and maintainers have collected in form of documentation has already been of great use for our project, and we are contributing back by updating documentation and creating issues, when we find disparities between the behaviours of our own server implementation and their server implementation.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#xmpp","title":"XMPP","text":"
Much like XMPP, I have decided to make polyproto an extensible protocol.
I am of the opinion that XMPPs biggest downfall is how many extensions there are, and that a server aiming to be compatible with other implementations of XMPP-based chat services should aim to implement all of the XEPs to be a viable choice.
polyproto is actively trying to circumvent this by limiting polyproto extensions (P2 extensions for short) to
-
either be a set of APIs and behaviours, defining a generic(!) version of a service. A \"service\" is, for example, a chat application, a microblogging application or an image blogging application. Service extensions should be the core functionality that is universally needed to make an application function. In the case of a chat application, that might be:
-
Defining message group size granularity: Direct messages, Group messages, Guild-based messages
- Defining what a room looks like
- Defining the APIs and behaviours required to send and receive messages
- Defining the APIs and behaviours required to perform commonly sought after things, such as reacting to a message with an emoji
- etc.
The goal is that all different polyproto-based chat applications should then implement this shared behaviour. Of course, developers may absolutely add their own behaviours and functionality which is perhaps exclusive to their specific implementation. Core functionality remains commonly defined however, which should make all polyproto-based chat applications interoperable in these defined, common behaviours.
- or describe a major technological addition, which can be used in the \"requires\" section of another P2 extension. This \"requires\" section can be thought of like the dependency list of a software package.
Technological additions might be: - Defining APIs and behaviours needed to implement the MLS (Messaging Layer Security) Protocol - Defining APIs and behaviours needed to establish and maintain a WebSocket connection, and how to send/receive messages over this WebSocket connection.
By using clay-brick-sized building blocks instead of more LEGO-sized building blocks like XMPP does, we hope to mitigate this problem that we perceive, while still offering an extensible yet well-defined platform to build on.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#matrixelement","title":"Matrix/Element","text":"
Matrix is perhaps the closest we have yet gotten to federated chat software aimed towards a general audience. However, as a strong believer in user experience - especially how first impressions impact new, non-technical users, I believe that Matrix falls flat in this regard. A lot of peoples first experience with Matrix is the infamous \"Could not decrypt: The senders device has not yet sent us the keys for this message\". The protocol and its sub-protocols are vast and complicated and use bespoke cryptography protocol implementations such as Olm and Megolm, which, in the past, has already been the cause of high-caliber vulnerabilities (see: https://nebuchadnezzar-megolm.github.io/ and, more recently, https://soatok.blog/2024/08/14/security-issues-in-matrixs-olm-library/#addendum-2024-08-14).
Matrix is truly impressive from a technical standpoint. Its extremely low centralized architecture fills a niche which especially people already interested in technology seem to enjoy. However, this invariably results in the fact that user experience has to be compromised. It is of my opinion that while Matrix is relatively good at what it is doing, it is not a good fit to be a potential Discord replacement.
As for a comparison: We are taking a radically different approach to Matrix. Matrix aims for eventually-consistent federation of events using cryptographically fully verifiable directed acyclic event graphs, where as polyproto, and by extension Polyphony, prioritize usability above all, intentionally disregarding highly complex or novel data structures in favor of cryptographic verifiability through digital signatures and simple public key infrastructure.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#what-are-significant-technical-challenges-you-expect-to-solve-during-the-project-if-any","title":"What are significant technical challenges you expect to solve during the project, if any?","text":"
Currently, our trust model acknowledges, that a users home server is able to create sessions and session tokens on the users' behalf, and is thus able to listen in on unencrypted communications, or, in the case of a truly malicious admin, would even be able to send messages on behalf of the user. This is not a novel problem, as it also affects all Mastodon ActivityPub servers in existence. Given that this potential abuse risk has not been a large issue in the Fediverse, we expect this to also not be a major problem. However, I would like to find additional mitigations or even a solution for this problem during further development of polyproto.
Another area that will likely need more work is my current design for how to connect to multiple servers at once: Currently, I expect every client to hold a WebSocket connection with each server that they are communicating with, at once. Depending on the amount of traffic, this could lead to constantly high resource consumption for clients. If this turns out to be the case, I am sure that we can find plenty of software- and protocol-side adjustments and improvements to implement - though it is still a potential technical challenge.
My last major area of concern is how well transmission and de-/serializing of the X.509 based Identity Certificates will work. I am optimistic about this however, since the X.500 series of RFCs are extremely well documented and already deeply explored, so that even if challenges arise in this area, I am certain that there is enough literature on the exact problem we might be facing, and enough people to ask/talk to.
"},{"location":"blog/2024/10/14/nlnet-grant-application/#describe-the-ecosystem-of-the-project-and-how-you-will-engage-with-relevant-actors-and-promote-the-outcomes","title":"Describe the ecosystem of the project, and how you will engage with relevant actors and promote the outcomes?","text":"
As the commercialization of Discord.com steadily increases, it is becoming clear that people are looking for a usable alternative. This is an audience that we are hoping to capture. Our Polyphony Chat service is Discord API compatible, so that actors may use the Polyphony client to interact with both Discord.com and polyproto-chat-based instances, and that existing bots and automations could potentially be ported over very easily. This essentially gives people looking for a Discord replacement exactly what they are looking for, as there should be little to no additional concepts, behaviors or patterns that users have to learn or re-learn to use our service.
As previously touched on, we are blessed to already have made a great amount of connections to like-minded developers also working on similar projects, who are looking optimistically towards polyproto as the tool to use to federate. I also have received explicit permission from Spacebar Maintainers to promote our projects on their Discord Guild, which currently counts 3600 members.
"},{"location":"blog/archive/2024/","title":"October 2024","text":""},{"location":"blog/archive/2023/","title":"November 2023","text":""},{"location":"blog/category/polyproto/","title":"polyproto","text":""},{"location":"blog/category/updates/","title":"updates","text":""},{"location":"blog/category/x509/","title":"X.509","text":""},{"location":"blog/category/chorus/","title":"chorus","text":""},{"location":"blog/category/polyphony/","title":"polyphony","text":""}]}
\ No newline at end of file