This Project started back when I was actively using & maintaining a public TeamSpeak-Server.
TeamSpeak is a proprietary voice-over-Internet Protocol application for audio communication between users on a chat channel, much like a telephone conference call. Managing it is unfortunately pretty inconvenient, logs are weird, the API is telnet-only, there are no good moderation tools or analytics, and tools for GDPR compliance are also subpar.
I started out adding more and more features to my personal TeamSpeak-Server. Some of these small tools can be found on my GitHub, most are terribly written and outdated though. Eventually, other providers of TeamSpeak-Servers were interested in running those solutions themselves.
After a careful amount of planning and research, I started development on TeamSpeak.gg. It had to provide scalable analytics, gamification, moderation tools to multiple VOIP-Servers.
Architecture
TeamSpeak-Severs unfortunately only provide events via an SSH/Telnet socket, same for commands. If a TeamSpeak query interacts with a user via chat and the query disconnects, the chat shows a “this user went offline” message. No idea why, one of the many weird quirks of TeamSpeak.
This means a central way of controlling access to query commands sent to customer servers was required. Several stateful daemons are set up to keep a persistent connection with multiple servers, keeping track of their respective rate limits and sending events to serverless webhooks that process all events and interact with the database. Redis is used for quick access and caching as well as managing the user levels and leaderboards. Most information can easily be stored in a KV-DB because every interaction with a TeamSpeak-Server requires a UUID anyway. TimescaleDB provides time-series analytics and some stuff that’s not suited for Redis.
This sure isn’t the perfect solution, but it utilizes some tech I wanted to use and experience at the time. TimescaleDB turned out to be quite useful. I was quite unfamiliar with the workings of time series data at the time and having the familiar, relational component in there helped me a lot. To provide better scalability, using a serverless service like Timestream would have been better.
The idea was to abstract away all the weird quirks TeamSpeak has and provide a more pleasant experience to users, administrators, and developers. All actions could be queued through a REST-API instead of telnet, events could be served to custom webhooks and user data saved by TeamSpeak was combined with the metadata we aggregated.
All business logic is invoked via events. The daemon manipulates the state via a GraphQL API and is also subscribed to any new state. This removes complex state management in your API layer and defers it to bespoke business logic functions. Events can also be persisted so that the entire history of state is available for observability. The daemons give the “commands” and then “queries” (subscribes) for the new state while events invoke the business logic behind the scenes.
Videochat
A big selling point for our tool was our video chat. It is seamlessly integrated with existing TeamSpeak voice channels. You could open your server's video chat URL in the browser of your choice and, after successful authentication, could share their camera and screen with everyone in the same voice channel. If someone left a voice channel they would automatically be removed from the corresponding video channel and joined the new one corresponding to their new voice channel.
This way we could rely upon the already existing access and permission control of TeamSpeak-Servers and only needed to build our video channels on top of that. We decided to run a custom fork of jitsi-meet to accomplish this. This allowed us to manage meeting participants and permissions based on events and also allowed us to disable potentially malicious P2P settings.
Each participant will have a randomly generated key which is used to encrypt the media using WebRTC Insertable Streams. The key is distributed with other participants (so they can decrypt the media) via an E2EE channel which is established via TeamSpeak’s already existing chat channels.
Ranking
As part of your gamification features, users were able to gather XP with their activities on each TeamSpeak server. To encourage competition we wanted to show each user's rank on the leaderboard. A Redis sorted set perfectly suited this use-case. They allowed us to add, remove, or update users XP in a very fast way (
O(log(N))) because elements are stored in order and automatically reinserted at the right position if an update happens. In order to have leaderboards “reset” each month we used a CycleKey together with the base key.
Cycle | Example Key | Key Function |
yearly | y2020 | y${time.getFullYear()} |
weekly | w2650 | w${Math.floor((time.getTime() + 345600000) / 604800000)} |
monthly | y2020-m05 | function above + m${time.getMonth()} |
daily | y2020-m05-d15 | function above + d${time.getDate()} |
every 3 months | y2020-q1 | y${time.getFullYear()}-m${Math.floor(time.getMonth() / 3)} |
every N days | dN-12 | dN-${Math.floor(getDaySinceEpoch(time) / N)} |